SAN FRANCISCO–(BUSINESS WIRE)–Ziliz, the founders of the Milvus open source project, today announced a major contribution to the Milvus 2.1 release. The added feature further narrows the gap between data pools, eliminates data hoarders, and offers performance and availability improvements to address common developer concerns. Milvus is one of the world’s most advanced vector databases capable of handling massive amounts of structured and unstructured data to accelerate the creation of next-generation data structures.
A phase-in open source project under the LF AI and Data Foundation, Milvus is designed for scalable similarity search and is used across industries. It incorporates a distributed architecture and can easily scale as data volumes and workloads grow. Highly scalable, reliable and extremely fast, Milvus supports DML operations (add, delete, update) and near-real-time vector lookups at trillion-byte scales.
With this 2.1 update, Milvus has significantly improved its performance, reducing search latency on million-scale datasets to five milliseconds and further simplifying deployment and operations workflows.
Closing gaps and improving performance
Machine learning generates tons of scalar and vector data every day. By introducing more scalar data types, Milvus 2.1 fills this critical gap between data pools.
“Data caches can now be better integrated and connected, allowing companies to realize the full potential of their data,” said Milvus project supervisor Xiaofan James Luan, who is also Ziliz’s director of engineering. “When it comes to unstructured data, the solutions offered by established companies in the industry are usually additional capabilities or tools in a legacy database management system, and Milvus has been built around unstructured data from day one and now offers more built-in capabilities to unlock.” more powerful and integrated data processing.
Ziliz’s contributions to version 2.1 specifically include:
- Overall performance increase including reduced latency; greatly improved throughput for small NQ applications such as reverse image search and intelligent chatbot; support for multiple memory copies for small tables to increase throughput; and 2x search performance.
- Improved scalar data processing which adds Varchar to supported data types and supports the creation of indexes on scalar data, taking hybrid search to a more intuitive level.
- Production-level improvements and increased availability, with clearer monitoring metrics for monitoring, simpler and more versatile deployment options, including embedded Milvus for simple deployment and Ansible for offline deployment, integration that supports Kafka as a log store, and improved security, supporting password protection and TLS connection.
- A developer-friendly ecosystem in development, including more tutorials for building real-world applications, connecting Milvus with the open source vector data ETL system Towhee; and it adds Feder, an open source tool that helps Milvus users choose the index that best suits their application scenario by visualizing the vector similarity search process.
In addition to the listed integration and security features, Milvus will provide more features necessary for modern security mechanisms, including ACLs (access control lists) and advanced encryption methods.
Commitment to open source ecosystems
“As a data infrastructure for unstructured data, Milvus is revolutionary because it handles vector embeddings, not just strings. In the future, Zilliz, a company founded by Milvus developers, aims to build an ecosystem of solutions around Milvus, and some projects that will contribute to this have already appeared, including Towhee, our open source vector data ETL system. , and Feder, an interactive visualization tool for unstructured data. With Milvus 2.1 and the new demos, users can see how these products can come together to solve many problems related to unstructured data,” added Luan.
Ziliz is committed to the developer community and will continue to contribute to open source projects such as Milvus. The company’s technology has a wide range of applications, including new drug discovery, computer vision, recommendation engines, chatbots and much more.
Ziliz is a leading vector database company for production-ready AI. Created by the engineers behind Milvus, the world’s most popular open source vector database, Ziliz’s mission is to unlock data insights using AI. The company develops next-generation database technologies to help organizations rapidly develop AI/ML applications and unlock the potential of unstructured data. By removing the burden of complex data infrastructure management from its users, Ziliz is committed to empowering AI for every corporation, every organization, and every individual.
Headquartered in San Francisco, Zilliz is backed by a number of prestigious investors including Hillhouse Capital, Aramco’s Prosperity7 Ventures, Temasek’s Pavilion Capital, 5Y Capital, Yunqi Partners, Trustbridge Partners and others. Ziliz technologies and products help more than 1,000 organizations worldwide easily build AI applications in a variety of scenarios, including computer vision, image acquisition, image analytics, NLP, recommendation engines, targeted ads, personalized search, intelligent chatbots, fraud detection, network security, the discovery of new drugs. , and much more. Learn more at zilliz.com or follow @zilliz_universe.