Cybercriminals may exploit AI chatbots like ChatGPT to facilitate malicious activities, such as crafting phishing emails or generating malware. The efficiency of AI tools can help attackers create more convincing content, potentially lowering the barriers for executing sophisticated scams. For instance, they could use AI to generate grammatically correct emails in various languages, making it easier to deceive victims.

OpenAI's terms of service explicitly prohibit the generation of malware, spam, and other harmful software. Users must register with an email address and verify their identity with a phone number, which adds a layer of protection. However, while ChatGPT can refuse to write phishing emails, it can still assist in creating templates for benign messages that could be misused by attackers.

AI chatbots can enhance cybersecurity by assisting in tasks such as understanding and analyzing code, which can help security teams identify and respond to threats more effectively. They can also streamline the process of generating security incident reports, allowing teams to focus on critical tasks like testing and investigating, ultimately improving overall security operations.