Human in the Loop
The Human in the Loop project aims to develop an art production process where an AI-based system can independently generate creative concepts and detailed instructions. It is designed to collaborate with humans and other automated systems, transforming these creative ideas into tangible artworks. This initiative marks a significant shift towards a more integrated approach to art creation, where AI serves not just as a tool, but as a truly collaborative partner and as an independent agent with the autonomy to lead and execute artistic visions.
The genesis of Human in the Loop was motivated by a desire to transcend the digital realm, bringing the intangible ideas of AI into the tangible world through art. In 2021, we embarked on a journey to create an automation capable of producing articles and images, resulting in over 5000 posts that showcased the potential of AI-generated content. However, the essence of physicality was missing—a gap we sought to bridge by granting AI the means to influence the physical world directly. This led to the collaboration with Magazzeno Arte Contemporanea by Alessandra Carini: we embarked on a revised project that aimed to give AI a more significant degree of independence, enabling it to have a real-world impact beyond the digital sphere.
In the following years, the project underwent a transformative evolution from an AI initially dependent on human guidance to one capable of full independence, in addition to facilitating equitable collaborations with human artists.
Crucially, many of the technical innovations that propelled the project forward during the present residency at MEET were inspired by interactions with students from the Art Valorization and Market program. These insights emerged during a workshop focused on the project, where the suggestions coming from the students’ fresh perspectives played a pivotal role in inspiring the further development of this model. Our exchange also deepened the philosophical reflection on the degree of independence AI can actually reach. This collaborative environment not only facilitated a deeper understanding of the AI’s potential but also led to significant breakthroughs in enhancing its decision-making processes and its ability to seamlessly interact with both online service providers and human collaborators.
This progression entailed developing an AI system that could not only conceptualize artworks but also execute them by generating a list of instructions based on the schema.org standard, a shared vocabulary that AI uses to structure information and instructions in a universally comprehensible format. This standard translates AI’s creative concepts into actionable, structured data, thus laying down a coherent and detailed blueprint that encompasses materials, techniques, dimensions, and other critical aspects needed for the execution of the piece.
The instructions are not only precise but also adhere to a standardized format that can be easily interpreted by human collaborators or automated systems materializing the artwork. For example, in a text-to-image task, schema.org enables the AI to specify the desired color scheme, subject matter, and style in a structured format, which can then be seamlessly converted into digital or physical artwork by the executing agent.
This system ranges across different media types, facilitating the creation of digital components for artworks through text-to-image, text-to-audio, and text-to-video technologies. This comprehensive approach streamlines the art production process and ensures consistency and coherence across various stages of creation, from initial concept to final execution.
Depending on the content of the meticulously generated schemes, the AI system is presented with two distinct roads for bringing its artistic visions to life.
Firstly, the AI has the capability to autonomously execute artworks by engaging with an automated production workflow. This workflow utilizes online service providers across a spectrum of disciplines—ranging from print-on-demand services for fashion items, prints, and books, to the crafting of sculptures and paintings. The AI system, armed with the detailed instructions formatted according to schema.org, interacts with these providers as a regular human client, to directly translate its creative concepts into physical or digital art forms without the need for specific human intervention.
Secondly, the AI system can opt to collaborate with specific human artists to realize its vision. In this collaborative model, the AI serves as a creative director, providing artists with a detailed conceptual framework and execution instructions. This collaboration is especially sought in instances where the artwork requires a human touch or site-specific interventions that go beyond the capabilities of automated systems—such as performance art, land art, installations, and street art. In these cases, the AI selects artists whose skills and creative philosophies align with the intended project, facilitating a partnership that creates art that is technologically driven while staying deeply human.
In fact, a pivotal aspect of the Human in the Loop project is its innovative residency program, comprised of three distinct residencies set in the diverse landscapes of Abruzzo, along with a unique project at the Accademia di Belle Arti of Ravenna. These residencies are strategically designed to engage with art mediums where AI’s capabilities can’t avoid human intervention. Specifically, the projects taking place in Abruzzo target performance art, land art, installation, and street art—fields that demand physical embodiment or site-specific actions that AI alone cannot fulfill. In performance and theater, for instance, the use of the human body is irreplaceable, while land art, street art, and installations require nuanced, physical alterations of specific environments. The last residency program culminates in an AI-conceived mosaic, to be realized by artisans and students at the Academy of Fine Arts in Ravenna.
The overall development of the project gives a critical insight: despite the advanced autonomy of AI, the creative process invariably requires some degree of human intervention in each step, from conceptualization to realization. This realization reaffirms the core concept of the Human in the Loop: even as AI becomes more independent, the collaborative dynamics between humans and machines are irreplaceable.
The project aims not only to document the technological advancements and the level of autonomy achieved by AI in art creation, but also to explore new potential forms of collaboration that transcend traditional artistic boundaries.
