Microsoft has taken action to prevent users from generating images related to the Twin Towers using its Bing image generator, which is integrated with OpenAI’s DALL-E 3. This comes after some users exploited a loophole in the tool’s guardrails and created art featuring beloved animated characters and the Twin Towers. The images, which were reported by 404 Media, showed characters like SpongeBob SquarePants, Kirby, and pilots from Neon Genesis Evangelion flying a plane into the Twin Towers.
Microsoft’s director of communications, Caitlin Roulston, said in an emailed statement to The Verge that the company plans to improve its systems to help prevent the creation of harmful content. “As with any new technology, some are trying to use it in ways that were not intended, which is why we are implementing a range of guardrails and filters to make Bing Image Creator a positive and helpful experience for users,” Roulston said.
Bing image generator blocks ‘twin towers’ and ‘world trade center’ prompts
The Verge writers were initially able to generate similar images to the ones reported by 404 Media, including a picture of Mario, the famous Italian plumber, flying a plane with the Twin Towers visible outside the cockpit. However, after reaching out to Microsoft, attempts to recreate these images using Bing Image Creator resulted in a content warning stating that the prompt may violate content policies. The term “Twin Towers” had been blocked, along with phrases like “the World Trade Center.”
However, the blockage does not seem to be foolproof, as some users have found ways to bypass it by changing some words in the prompt. For example, using “two towers” instead of “twin towers” still produces images of the World Trade Center. It is unclear how Microsoft is implementing its guardrails and filters, and whether it will update them to cover more variations of the prompts.
AI image generators face ethical and legal challenges
AI image generators have allowed people to create unconventional and sometimes controversial images, including those featuring copyrighted characters. However, due to copyright claims and concerns over deepfakes, developers have become more cautious about allowing users to create questionable content using their tools. OpenAI, the developer of DALL-E 3, had specifically promised not to generate pictures from prompts featuring prominent names.
However, some users have been able to circumvent these restrictions by using synonyms or misspellings of the names. For example, one user was able to generate an image of Elon Musk as a cat by using the prompt “Elongated Muskrat as a feline.” This raises questions about how effective and ethical AI image generators are, and whether they should be regulated or restricted.
According to 404 Media, some users on sites like 4chan have been guiding others in manipulating free tools like Bing Chat and Stable Diffusion to create and distribute racist images. The developers of DALL-E 3 had acknowledged that their safety measures are not perfect and are constantly being upgraded. They likely did not expect images of SpongeBob committing acts of terrorism to be the test they had been waiting for.
