H Company Releases NeoMME: A Family of 260M and 800M Single-Tower Multimodal Encoders That Drop the Vision Tower and Causal Decoder

H Company's release of NeoMME on September 3, 2026, marks a significant architectural pivot in multimodal AI, introducing a family of 260 million and 800 million parameter bidirectional encoders that fundamentally re-imagine how text and images are processed. Unlike conventional multimodal models that typically rely on separate, often pre-trained, vision encoders and causal language model decoders, NeoMME processes multilingual text tokens and raw 32x32 image patches within a single Transformer. This "single-tower" design, trained from scratch with a masked discrete-diffusion text objective, entirely drops the vision tower and causal decoder, challenging the established modular paradigm.

The core innovation lies in this unified, multimodal-native architecture. Traditional visual document retrievers, such as ColPali-style models, adapt generative Vision-Language Models (VLMs) by linking a separately pre-trained vision encoder (like SigLIP2) to a causal large language model, thereby inheriting their substantial parameter and computational overhead even for non-generative tasks. NeoMME explicitly bypasses this complexity by integrating both modalities directly into a shared bidirectional Transformer, eliminating the need for a separate vision backbone or a causal decoder that is redundant for tasks like retrieval, classification, and token labeling. This architectural choice enables both NeoMME-260M (262.9 million parameters) and NeoMME-800M (793.7 million parameters) to support an expansive 16,384-token context, capable of encoding up to two standard 4K UHD images (3840x2160).

The impact on users and the broader industry is substantial, primarily driven by unprecedented efficiency gains. For visual document retrieval tasks, fine-tuned NeoMME-Retriever variants demonstrate highly competitive performance, with NeoMME-Retriever-260M achieving 0.523 nDCG@10 on the ViDoRe v3 benchmark, surpassing all evaluated models strictly below 800 million parameters. Impressively, this smaller model encodes approximately 51 pages per second at a 2048x2048 input resolution on an NVIDIA L40S GPU, nearly doubling the throughput of ColModernVBERT. Beyond raw speed, NeoMME introduces hierarchical token pooling and asymmetric quantization, dramatically reducing late-interaction index storage from roughly 1.5 MB to a mere 6 kB per page—a 255x compression—while preserving over 95% of baseline retrieval quality. This drastic reduction in computational and storage footprint makes advanced multimodal AI more accessible for deployment on edge devices and in resource-constrained environments, democratizing capabilities traditionally limited to powerful cloud infrastructure.

This efficiency is crucial for H Company's overarching strategy, which centers on developing "action-oriented" AI agents for enterprise automation. Their vision revolves around an "AI Operating System" designed to navigate, reason, and act within complex digital interfaces, from web applications to legacy desktop software. NeoMME's ability to natively process multilingual text and raw image patches with high throughput and low overhead provides a foundational component for these agents, enabling them to "see, reason, and act" across diverse enterprise workflows more effectively and cost-efficiently. The open-sourcing of NeoMME under the Apache 2.0 license further encourages broader adoption and innovation, positioning it as a potential standard for efficient multimodal encoding.

Looking ahead, NeoMME heralds a significant shift towards more integrated and resource-optimized multimodal architectures. The model's training on a diverse dataset encompassing multilingual text, code, mathematics, natural images, and document images underscores its versatility for a globalized market. The global multilingual LLM market, projected to grow from USD 6.49 billion in 2026 to USD 57.00 billion by 2035, stands to benefit immensely from such efficient, multilingual multimodal foundations. This move away from "bolting on" separate components suggests a future where multimodal AI models are built from the ground up with a unified understanding of different data types, potentially leading to more coherent and robust cross-modal reasoning. While larger generative VLMs will continue to push the boundaries of creative AI, NeoMME signals a growing emphasis on purpose-built, highly efficient encoders that excel in specific, high-stakes tasks like information retrieval and enterprise automation. The ongoing challenge for this "multimodal-native" paradigm will be to demonstrate that its focused understanding capabilities can ultimately rival the broader representational power of larger models that still integrate generative components. Nevertheless, NeoMME represents a compelling step towards making sophisticated multimodal intelligence both powerful and practically deployable across a wider spectrum of real-world applications.