AI Supporting Creative Industries

 NYC Media Lab (NYCML) and Bertelsmann unveiled  the latest cohort joining the AI & the Creative Industries Challenge, a nine-week program in which teams explore new ways to use artificial intelligence (AI) to create digital content and reach new audiences for three Bertelsmann companies: Fremantle, Penguin Random House, and BMG. The teams are tasked with addressing how AI will impact these important creative industries. 

This ongoing partnership, NYCML’s third project with Bertelsmann, will continue to build on new business frontiers enabled by technology. The four selected teams, from around the globe, come from various multidisciplinary backgrounds. 

“Bertelsmann is deeply involved in experimenting with AI. Our Bertelsmann team will broaden their perspectives on this technology by teaming up with NYC Media Lab to work with this new cohort,” said Bertelsmann, Inc. Senior Director of Human Resources Freddie Helrich.

“Ensuring that the newest technologies are applied to the creative industries we have held near and dear to our hearts is a perfect example of why industry - Bertelsmann in this case - and academia should work hand in hand,” said Sayar Lonial NYC Media Lab Interim Executive Director and Associate Dean for Communications & Public Affairs at NYU Tandon.  “We are  excited to work with Bertelsmann to see how AI can support communications in all forms.”

The AI & the Creative Industries Challenge Teams

Author AI from Abelana VR   
Mik Labanok, Denis Chernitsyn
Brooklyn, New York

Author AI, is a tool for publishers, producers, and digital marketers to create interactive, virtual experiences based on their characters and authors.  Author AI represents the lore of a property through virtual assistants immersed in a theme-based environment and connected to a variety of third-party resources. 

Abelana VR is a developer and publisher of virtual applications for education, training, and other knowledge-driven content. Its main production is focused on online multiplayer experiences created with a VR-first approach and designed to fit across a wide range of ecosystems, including VR, AR, mobile, and web.

Smartplayr from SAOViVO & Axle.ai
Elisa Hecker, Emiliano Billi, Nicolas J. Russo, Sam Bogoch
Buenos Aires (Argentina) and Boston (USA)

Smartplayr repurposes existing media into live streams while keeping it current and relevant using AI.  They are leveraging AI in a number of ways to automate and optimize live streaming: Face detection for better screen composition and dynamic chyrons; Adaptive user interface to fit different aspect ratios; Scene detection to facilitate the reuse of pre-recorded content; Live transcription of breaking news to highlight important information. 

They are a multidisciplinary team composed of two companies utilized by current newsrooms: SAOViVO, an open source software that turns video playlist into a live stream, and Axle.ai, a powerful media asset manager (MAM) and publishing solution. 

Theater of Latent Possibilities from Speculative Devices + Cohab Labs
Ash Eliza Smith, Ryan Schmaltz, Robert Twomey, Jinku Kim, Patrick Coleman
Lincoln, Nebraska

Theater of Latent Possibilities focuses on the construction of workflows for pre-production and performance for TV, Film, and Theater. utilizing generative AI with sound, visuals, and writing. Their“writer’s room” tool allows for worldbuilding and co-creation with generative AI. Their system surfaces unique moments, unexpected connections, and latent narratives present in input datasets. At runtime, they employ these generative techniques for real-time performance—creating live, immersive, participatory experiences that hinge on the improvisatory dynamics of human-machine co-authorship.

The team is composed of artists, writers, musicians, engineers, and business practitioners exploring the frontiers of worldbuilding, co-creation, and generative AI in media and performance. They have published and performed our work in a range of international venues spanning academic conferences, arts festivals, and research institutes. 

Wavetable
Johann Diedrick, Sylvia Ke
New York City

Wavetable is an innovative web-based platform for sound and music production, offering a swift and efficient solution for professionals in the music, publishing, and film/TV industries. Using text-to-audio generative models, Wavetable empowers creators to articulate their sonic visions using natural language. It then transforms these descriptions into tangible audio outputs that can serve as preliminary placeholders before custom, polished audio content is crafted. Wavetable expedites the realization of creative concepts, substantially shortening project timelines, while also affording creators the freedom to develop audio content autonomously.

The team possesses a distinctive blend of industry experience spanning both the technical and creative realms. Their unique background enables them to pursue seamless integrations of AI-driven solutions into the established workflows and industry-standard products of the creative sectors. Their approach to product development is characterized by a design-first mindset, prioritizing creating AI tools that are not only powerful but also intuitive, versatile, and accessible

Program Details

Teams will work with mentors from Bertelsmann’s music, book publishing, film and TV production, digital and investment arms. NYC Media Lab colleagues and academic partners will also provide direction and feedback to the teams. The Challenge will conclude in December with an internal Demo Day, where teams will demonstrate their project outcomes and discoveries.

About Bertelsmann

Bertelsmann is a media, services and education company that operates in about 50 countries around the world. It includes the entertainment group RTL Group, the trade book publisher Penguin Random House, the music company BMG, the service provider Arvato Group, Bertelsmann Marketing Services, the Bertelsmann Education Group and Bertelsmann Investments, an international network of funds. The company has 165,000 employees worldwide and generated revenues of €20.2 billion in the 2022 financial year. Bertelsmann stands for creativity and entrepreneurship. This combination promotes first-class media content and innovative service solutions that inspire customers around the world. Bertelsmann aspires to achieve climate neutrality by 2030.

About The NYC Media Lab

The NYC Media Lab connects media and technology companies with both NYU Tandon and industry affiliates to drive innovation, entrepreneurship and talent development. Our interdisciplinary community of innovators from industry and academia allows our network to gain valuable insights, explore the potential of emerging technology and address the challenges and opportunities created by the rapidly evolving digital media landscape. Learn more at engineering.nyu.edu/nyc-media-lab.