27 stories tagged Multimodal
Researchers created Infinity-Parser2, an AI that can read and understand documents better than ever. It uses a special training method to handle all kinds of documents, from contracts to reports, in both English and Chinese. This could make document processing faster and more accurate for businesses and everyday users.
Researchers have developed a new unified multimodal foundation model that jointly models vision, language, world dynamics, and action generation. This advancement could lead to more capable robots and virtual assistants that can understand instructions, anticipate environmental changes, and execute precise actions over extended horizons.
Researchers have unveiled Gemma 4, a new suite of open-weight AI models that handle text, images, and audio together. These models are designed to be more efficient and better at reasoning, with sizes ranging from 2.3 billion to 31 billion parameters.
Researchers developed a way for AI assistants to highlight where they found answers in documents. This could make chatbots more trustworthy by showing their sources instantly.
A new paper highlights flaws in how we test AI models that handle text, images, and other inputs together. Current methods miss key aspects like understanding physical reality or combining different types of information. The authors propose better evaluation frameworks to address these gaps.
Researchers developed SciLens, an AI system that can verify scientific claims by analyzing both text and images. This tool could make it easier for scientists and the public to check the accuracy of research findings.
Researchers have traced the internal pathways through which AI models process and combine visual and audio inputs to reach decisions. The findings could lead to more transparent and reliable AI assistants and creative tools.
Researchers developed TIGER, a new AI system that improves accuracy in multimodal generation by tracking and correcting factual errors. This could make AI-generated images and text more reliable for everyday users.
Researchers have developed VFEAgent, an AI system that automates Finite Element Analysis (FEA) from images and text descriptions. This could make advanced engineering tools accessible to non-experts, speeding up design processes.
Hark, a stealthy AI startup, has raised $700M to create a universal AI interface that works across all your apps and devices. The company plans to launch its first multimodal models this summer, followed by custom hardware.
Google’s new Gemini Omni can create and edit videos from text, images, and audio using simple prompts. This could revolutionize how anyone makes video content without needing professional skills.
Researchers have developed a system called LatentRouter that can choose the best AI model for a specific image-based question before it even answers. This could make AI assistants much more efficient and accurate for visual tasks.
NVIDIA's new Nemotron 3 Nano Omni model supports long-context multimodal intelligence across documents, audio, and video. It is designed for developers to build advanced AI agents.
A new study explores source-modality monitoring in vision-language models, assessing their ability to track and communicate the origin of information. The research evaluates how models bind words to specific input components across 11 different models.
Researchers developed CognitiveTwin, a digital twin framework that predicts individual cognitive decline in Alzheimer's disease using multi-modal data. The model aims to provide accurate, fair, and robust predictions across diverse patient demographics.
Researchers developed AITP, an AI system that uses Multimodal Large Language Models to analyze traffic accidents and assign responsibility based on legal knowledge. This advancement could revolutionize accident investigations and insurance claims.
Researchers have introduced the first benchmark for extracting claims from multimodal social media posts, addressing a gap in automated fact-checking. The dataset includes text combined with images like memes and screenshots, challenging traditional methods.
Researchers introduce CFMS, the first fine-grained multimodal sarcasm dataset for Chinese social media. It includes 2,796 image-text pairs with triple-level annotations, advancing research in sarcasm detection.
Researchers introduce GIST, a new model that enhances spatial grounding in densely packed environments. The model addresses challenges faced by Vision-Language Models (VLMs) in cluttered spaces like retail stores and hospitals.
Hugging Face has published a comprehensive guide on training and fine-tuning multimodal embedding and reranker models using Sentence Transformers. This resource is aimed at developers looking to enhance their multimodal applications with state-of-the-art techniques.
Researchers introduce a dynamic data curation pipeline to better evaluate how LVLMs handle conflicts between visual and textual evidence. The study underscores the importance of models admitting when they lack knowledge.
TII has released Falcon Perception, a multimodal model combining text and vision capabilities. This open-source model sets a new standard for accessibility in AI.
Google's MedGemma 1.5 4B model now integrates high-dimensional medical imaging, anatomical localization, and multi-timepoint analysis within a single architecture. This update significantly expands capabilities to include CT/MRI volumes, histopathology, and complex EHR document understanding.
IBM has released Granite 4.0 3B Vision, a compact open-source model designed to analyze enterprise documents with high efficiency. This release bridges the gap between heavy multimodal models and the need for fast, local document processing.