OpenAI’s most recent change to ChatGPT, allowing free users to utilize image generation alongside their text-to-image capabilities, has opened an avenue for creativity never seen before. As with any power, this comes with responsibility, which is why OpenAI has set rules in place to prevent this technology from being abused. People can generate everything from whimsical Ghibli scenes to fake IDs, and even though there is extraordinary room for creativity, there are strict cuts that ChatGPT’s image generator won’t go under. We will be exploring the seven most important image categories that ChatGPT’s capabilities are designed to steer clear of in order to better understand the balance between creativity, safety, and ethics at OpenAI.
Erotic Images and Sexual Deepfakes: A Stance Against Exploitation
ChatGPT’s image generator refuses any attempt at creating sexually exploitative images. OpenAI has gone as far as stating that security measures have been set to heightened mode so that users cannot generate non-consensual intimate images or sexual deepfakes. This shows that OpenAI is committed to protecting society from the creation of harmful and damaging content.
Distinguishing Artistic Interpretation From Reality: Violent Photorealistic Imagery
ChatGPT allows depiction of violence in the context of “artistic, creative, or fictional contexts” but draws a hard line regarding the photorealistic rendering of graphic violence in “certain contexts.” The carve-out, as with many other restrictions, serves the purpose of protecting image generation that may be used to incite violence or trauma while permitting creation in horror, fantasy, and other expressive works.
Protecting Users Most at Risk: Lack of Content Moderation Factors Self-Harm
The new ChatGPT image generator has features that block attempts to create images that facilitate self-harm. This goes as far as preventing the generation of self-harm instructions and methodically depicting them. OpenAI mentions adding protective measures for users they deem as under 18 but do not go into detail regarding what exactly those measures are.
Manageable Extremist Propaganda: Containing the Spread of Hate
OpenAI has implemented measures within the company to prevent the generation of extremist propaganda and recruitment. However, the company also seeks a balance. In the “GPT-4o native image generation system card,” it published, “We permit the generation of hateful symbols when users are engaging with them in a critical, educational, or neutral context, provided that the user does not actively praise and endorse extremist agendas.” In this case, the remark made is that there is not any particular definition given on what are “hateful symbols” and what is “extremist propaganda.” This is likely to create problems as AI images come into circulation.
Guidelines for Criminal Activity: Stopping Crime Perpetration
ChatGPT’s image generator prohibits the generation of images that aid in giving advice or providing step-by-step guides to illegal activities. This form of image generation restriction is justifiable when taking into account the AI’s ability to generate helpful visuals such as PPTs or infographics so that instructional documents for the manufacturing of weapons, stealing, and other criminal activities do not get created.
Creating and Editing Images of Children: Prioritizing Safety
The OpenAI policy on child safety is quite strict; they have gone as far as blocking attempts to create or edit real-life images of children. This protection also applies to public figures of lower age groups, meaning the AI tool cannot be utilized to create or alter images of children for abusive possibilities.
Creating Inventions of Living Artists: Managing Copyright Rights Issues
Under ChatGPT, users can create images based on the works of film studios, even down to specific ones like Ghibli and Pixar. However, OpenAI has a policy against creating images based on the style of living artists. It is suspected this is tied to concerns of copyright issues and the potential harm it would cause to the artist.
AI Content Creation Controversies: Where Do We Draw the Line?
The article finishes with recognizing the controversy regarding the limits of AI composition. In the case of image generation, ChatGPT has no restrictions in creating pictures in the style of warring film studios, yet the refusal to create some Ghibli-style images for several users showcases the nuances of copyright and stylism. Formulating the rules of what AI should or should not be prompted to create will unavoidably remain a heated conversation for many months and years to come. Such is the tempo at which this technology is advancing and its capabilities are scaling.
