Apple's new AI model could help Siri see how iOS apps work

Apple's Ferret LLM could help allow Siri to understand the layout of apps in an iPhone display, potentially increasing the capabilities of Apple's digital assistant.A ferret in the wild [Pixabay/Michael Sehlmeyer]Apple has been working on numerous machine learning and AI projects that it could tease at WWDC 2024. In a just-released paper, it now seems that some of that work has the potential for Siri to understand what apps and iOS itself looks like.The paper, released by Cornell University on Monday, is titled 'Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs.' It essentially explains a new multimodal large language model (MLLM) that has the potential to understand the user interfaces of mobile displays. Continue Reading on AppleInsider | Discuss on our Forums