Dataroot Labs

AI Agent for Lead Management Summary For most b2c businesses the leads' outreach consumes a lot of resources and time, while the conversion rate requires intensive work to attract clients; Business optimization infers the goal of involving AI in those areas of the sales workflow that are reasonable, especially for initial communication to reduce the cost of leads processing; Our team developed a conversational AI agent capable of performing voice calls and chat interactions. The agent follows various conversational behaviors, adapts dialogue strategies based on client responses, and collects information to simplify the decision process and further workflow; The developed solution helped to significantly decrease the price of call center support while increasing leads coverage; Tech Stack Python OpenAI Cohere Anthropic Deepgram ElevenLabs Twilio Retrieval-Augmented Generation (RAG) Milvus Vector Database Prompt Engineering AWS Tech Challenge Natural Voice Interaction. The AI Agent must talk with a natural, human-like voice to avoid sounding robotic, which is typically associated with spam calls. Low Latency Communication. When handling the voice calls, the LLM-based agent must have an overall latency as speaking to a real person. Dynamic Conversation Management. Providing clear and concise instructions to the assistant to manage conversations effectively, dynamically changing states based on client responses. Information Gathering. Efficiently gather information about leads and their properties while maintaining conversational flow. Solution Used ElevenLabs for advanced text-to-speech synthesis, providing the assistant with a natural and friendly voice that reduces the likelihood of being perceived as spam. For speech recognition, the solution includes a self-hosted version of a Deepgram to ensure low latency and high quality of a solution. A special decision maker model such as OpenAI 4o-mini, is used in the system to adjust the flow of the conversation in real time. This model analyzes interactions and determines when to change dialogue strategies to ensure smooth transitions and appropriate responses. Implemented LLM-based entity recognition to extract key information about leads and their properties during conversations, and automatically process and log data for follow-up interactions. Enabled the agent to dynamically retrieve relevant information during conversations, enabling more accurate and contextually appropriate responses. Impact The agent significantly reduced the amount of time sales managers spent on the leads outreach by automating initial communications and routine tasks such as taking notes, sending follow-ups and meeting links. Automating of lead interactions led to reduced operational costs and increased throughput by optimising business processes. The agent became part of the SaaS platform, helping similar companies with operations of their sales manager and call center operations.  Read More

AI Robotics and Simulation for Enhanced Quality Control Summary An inventive client startup is revolutionizing the use of AI robotics in manufacturing. The company focuses on high-mix production, which involves making different types of products with various specifications. They intended to develop a simulation to identify product’s deformations using sensors to scan details and compare them with perfect parts. Therefore, we employed this solution to deal with some high-mix manufacturing problems and to increase accuracy and productivity in processes. The team used the NVIDIA Omniverse Isaac Sim platform to make an advanced simulation, with the integrated client’s robot prototype created in Onshape. A critical component was point cloud comparison using CloudCompare software for accurate and fast identification of discrepancies in manufactured parts. Tech Challenge The first issue for DataRoot Labs was to bring the client’s robot prototype into Isaac Sim and keep its differentials, such as dynamic joints and physical properties. Apart from this, it was essential to set up a hyper-realistic scene and code the flow of imitation with great care. This setup should ensure that each robot goes through every step by specified instructions. Efficient part point cloud comparison involved isolating Inspected components from other objects in the point cloud, especially the vise that holds them while rotating. Additionally, all noises from the sensor that originated from the LiDAR device have to be cleared away. The resultant scan comparisons should be vivid enough so that any deformity could be detected at a glance. Tech Stack C++ CUDA Python Nvidia Omniverse CloudCompare Solution We started by bringing the robot prototype into the NVIDIA Omniverse Isaac Sim. This highly sophisticated simulation allowed us to design and test robotic interactions within secured dynamic environments, ensuring system accuracy before physical implementation. Every joint was inspected to keep the robot’s dynamic and physical attributes intact. The joints must have the right tree-like structure for accurate robotic movements. One factory-made part is picked up by one robot and put into a vise held by another robot. After that, the vise closes around the detail and rotates, while the LiDAR sensor scans it. The rotation aims to capture all dimensions and features on the part with a 3D point cloud. ?We process the point cloud data using CloudCompy, a tool based on CloudCompare to separate the vise from all other parts of the point cloud so that only the part itself remains focused upon without its surrounding. ?Finally, we compare the processed point cloud against a perfect, predefined model. ??This way, if any misshapes or defects occurred during manufacturing, it’s easy to identify them. ?All detected deviations can be corrected unmistakably, which reduces waste and boosts production efficiency. Impact The introduction of this AI-powered system completely revolutionized the production operations for our client. By automating tasks in manufacturing settings they gained a significant boost in productivity and flexibility. Additionally, the manufactory cut down on both time and expenses related to prototyping and testing by simulating these processes using NVIDIA Omniverse Isaac Sim. Furthermore, the advanced processing of point clouds ensures that each component meets quality standards thereby improving the precision and reliability of the company’s products. Read More

AI-Powered Recommendation Engine for a Leading Health App Client Services: AI Solutions Development Summary The client is a popular weight loss app that focuses on changing behavior to adopt a healthy lifestyle and appearance. The app makes it entertaining to build new habits as part of a supportive community. Among other features, it enables users to listen to live sessions from wellness coaches, other experts, and leading voices. To increase user engagement and loyalty, the client imagined building a recommendation engine as a core function of the app. Its key task would be an effective matching system between user goals and preferences AND relevant live streams and health-related trending topics. Our team has developed an AI-powered recommendation engine that works with both audio and text formats. By leveraging a siamese-architecture-inspired network, the system considers multiple parameters to produce relevant and engaging content recommendations. Tech stack AWS ECS Fargate Lambda MSK (Kafka) OpenSearch Milvus Python Gunicorn aiohttp OpenSearch-py boto3 PyTorch Transformers Scikit-Learn Pandas Numpy Terraform AWS CloudFormation GitHub CodePipelines Tech Challenge Audio content, with its multifaceted attributes, poses a significant challenge for recommendation systems. The answer itself, along with the question's topic, relevance, provided value, and wisdom, all play crucial roles in the process of suggesting. All the inputs are usually recorded via phone. Thus, many things might contribute to overall audio quality — mic sensitivity, environmental noise, speech loudness, speed, pitch, etc. A key requirement is the real-time generation of recommendations influenced by the user's current activity. The system must ensure that new content is instantly available. This requirement eliminates the possibility of caching or pre-computed recommendations. The system should be designed to handle a growing user base. Services should be scalable and able to adjust quickly to changes in user volume. All recommendation systems have the cold start problem. It is usually related to new users who don't have historical activity. Such users should still be able to get hints right after signup and eventually should get a more personalized experience. Solution We trained an ML model to create embeddings for answers and users. The model features include audio transcription, question text, author description, and historical stats like listen rate, likes, comments, etc. Every day, the model is automatically retrained to tune weights and adjust embeddings based on the latest user's activity. All answers on the platform were moderated by humans and scored according to rules. Those scores reflected how good the answers were, how precisely they answered the topic, and how much value they gave to the listener. At the later stage of the project, we implemented an auto-scoring mechanism to reduce the amount of human work on the platform. We developed a comprehensive set of OpenAI ChatGPT prompts to evaluate the content score by the same criteria as humans did. Additionally, we developed an audio processing pipeline to analyze the audio quality based on a set of features like speech ratio, short-time objective intelligibility (STOI), perceptual evaluation of speech quality (PESQ), signal-to-noise ratio (SNR), etc. The system is primarily built using Python tech stack and scalable databases and is deployed on AWS Cloud. All the services are running on Lambdas and ECS. External integration with the app is performed via Kafka. Utilizing the OpenSearch and Milvus databases allows the creation of on-the-fly recommendations of 20-50 items in ~100-300ms time. Impact The recommendation engine serves as a key to the app's functionality ensuring the increase in daily active usage. Customers can easily find relevant content, be it audio clips, podcasts, chats, or trending topics. Thanks to a truly curated experience, millions of users enjoy better content that impacts their lives positively. Due to robust architecture and infrastructure design, the app can sustain supporting a large user base without latency issues. The consistent performance, great usability, and content relevancy ensure that the company consistently finds itself among the top players in digital health and weight management app ratings. Read More

AI-powered Market Research Agent for B2B Sales Client Services: AI Solutions Development Summary The client is a B2B software company offering IT consulting and development services. Their customers praise their work for exceptional technology expertise and continue to choose them as a preferred partner for IT-related tasks. Most of their new clients traditionally come from referrals and a personal network while outbound sales had limited results. To build a diversified and predictable outbound revenue stream, the client decided to develop a platform that would automate prospect searching and outreach. The tool would not only replace most of the existing capabilities of the human market researcher on the team but also significantly increase prospect sources and their volume. The built system would need to automatically find relevant prospects based on a defined customer profile, look up fresh information on each prospect, and create personalized outreach campaigns based on the findings. For example, a cold email campaign targeting companies with a certain position on their website that would also include insights from recent company news published on their blog. DataRoot Labs successfully developed an AI-driven Market Research Agent for the client. This solution generates a daily list of relevant prospects based on pre-defined filters, provides the latest insights on each company, and crafts personalized outreach campaigns. As a result, the client enjoys a substantial increase in response rate and sales calls, with dozens made each month. Tech Stack Python AWS OpenAI Postgres HubSpot API Woodpecker API Gmail API PhantomBuster Google Sheets API Slack API Tech Challenge To find suitable prospects, we had to identify relevant databases containing up-to-date information on potential customers. Therefore, the primary challenge was efficiently gathering information from various private data providers and other research solutions. Additionally, ensuring the accuracy and relevance of the extracted insights presented a significant hurdle. Another envisioned functionality was enriching the shortlist of prospects with relevant insights. The Agent can search and add insights based on publicly available data on each company — website information, press releases, articles, posts, etc. The final solution should work as a fully Autonomous Agent capable of making outreach through platforms for sending emails, including personalized sequences to drive engagement with prospects. Finally, the solution have to integrate with the sales team's existing toolkit through APIs, including email marketing, contact verification, CRM systems, and LinkedIn. Solution The main task of the Market Research Agent is gathering, combining, and filtering prospects from multiple sources. The Agent collects unstructured data about companies that may be interested in IT services using web search, scrapping tools, and data providers' APIs. Next, each company is comprehensively investigated. This includes analyzing the services and products listed on the company's website, the founders' contact information, the headquarters and office locations, etc. Based on the results of the investigation, irrelevant companies are automatically removed. With a full understanding of the potential client, the outreach strategy takes its course. The Agent finds the decision-maker among all company contacts, selects contact methods, writes message sequences, and sets up automated outreach with Woodpecker for email and PhantomBuster for LinkedIn. To simplify statistics collection, the Agent Analytics system was developed. All important conversions on each step are aggregated in real-time, allowing the sales department to enhance predesigned campaign parts and adjust prospect sources. Market Research Agent was also successfully integrated with the IT company's HubSpot CRM, which prevents doubling contacts and, along with Agent Analytics, ensures the clarity and visibility of the outreach results. Impact Implementing the Market Research Agent subtracted the time-consuming operations by automatically and continuously populating the CRM, ensuring a steady stream of prospects. Within the 3 months experiment window, it resulted in a 150% increase in monthly sales calls and doubled the number of monthly contracts signed. The market research department's processes have transformed from manual analysis and data entry to managing an AI-powered Agent who generates new prospects. This has upgraded the team's skills and increased the productivity of the entire sales department. By automating a great part of the sales process, the company set itself apart in a competitive market, reaching potential clients faster and more efficiently through personalization and advanced filtering. The end result was an uptick in revenue growth, ultimately achieving long-term sustainability and success. Read More

Digital Transformation of a Large Telematic Service Client Services: AI Solutions Development Summary Our client is a telematics service used for truck fleet management. Technically, their solution is a mix of sophisticated software and hardware for collecting and analyzing critical information about vehicle and driver parameters in real-time, allowing customers to manage their fleet with a high degree of efficiency and at the lower cost. The client strove to augment driver's safety and security and enhance the driving experience by digitally transforming its obsolete software into a modern platform powered by AI, able to track the large number of fleet vehicles in real-time. Our team designed and implemented different parts of the final product, including APIs, web-interfaces and mobile apps, with the main challenge to create the system which receives the data from hundreds of thousands of monitoring devices in real-time, working 24/7. Tech Stack Akka Apache Spark Cassandra GlusterFS Kafka PostgreSQL Scala Tech Challenge Tracking a large vehicle fleet requires integration of large amounts of sensor data into a single pipeline. Making tracked data available to compliance units within seamless interface, at a lower cost. Data has to be processed and accessible in near real-time fashion. Solution All the data is stored in the scalable cluster in a fault-tolerant manner (Cassandra). The mobile app has a real-time access to the information from each connected monitoring device. System automatically produces reports for each fleet vehicle, via collecting data from analytical indicators built by our team. Huge amounts of data are processed fast due to Apache Spark cluster; micro-services were built as an isolated Docker containers. Metrics gathered in real-time are numerous and detailed (i.e. number of stops, start and end time of the journey, mileage, speed, idle time, route history, geofences, fuel consumption, engine temperature, etc.) They are used to create automatic alerts about accidents, critical situations, theft and other situations. Impact We have developed a sophisticated platform for collecting and analyzing critical information about vehicle and driver parameters in real-time. The final solution allowed customers to manage their fleet with a high degree of efficiency and at a lower cost. The platform receives the data from hundreds of thousands of monitoring devices in real-time, working 24/7.  Read More

Knowledge AI for Oracle NPC in MMORPG Client Services: AI Solutions Development Summary The client is developing a massively multiplayer online role-playing game (MMORPG) set in a vast, immersive fantasy world. This world is rich in lore, characters, quests, and items, offering players a deep and engaging gameplay experience. The game boasts a dynamic environment where players can interact, form alliances, and embark on epic adventures. The client envisioned an innovative feature for their MMORPG: an Oracle Non-Player Character (NPC) that players could approach and consult. This Oracle would serve as an in-game knowledge base, answering players' questions about the game's lore, mechanics, or personal progress. Notably, the Oracle's knowledge would be gated, revealing information only in line with a player's in-game achievements and progress. A sophisticated knowledge system was implemented to bring the Oracle NPC to life. At its core, the method utilized the LLaMa 2 as the primary LLM. Milvus, a robust vector database, managed and retrieved vast amounts of in-game data. The embedding of this data for efficient querying was achieved using the SetFit algorithm, ensuring that players received accurate and contextually relevant answers from the Oracle. Tech Stack Rust Python C++ GGML HuggingFace PyTorch Milvus FAISS Kafka Kubernetes AWS Docker Tech Challenge Scale: MMORPGs, by nature, cater to many players simultaneously. The Oracle NPC, being a unique feature, was expected to be frequently consulted by thousands of players. This meant the system had to handle a massive volume of queries in real-time, ensuring each player received accurate and timely responses without any noticeable lag. Latency: Even a slight delay can disrupt the immersive experience in gaming. The challenge was not just to provide accurate answers but to do so with minimal latency. Integrating an LLM like LLaMa 2 with the game's infrastructure while ensuring that players' response time remained imperceptible was a significant hurdle. Personalized Knowledge: The Oracle's knowledge had to be adaptive, revealing information based on a player's progress. This meant the system couldn't simply provide generic answers. It had to recognize each player's achievements, unlocked content, and current game status, tailoring its responses accordingly. Privacy: While cloud-based solutions are prevalent, the client wanted to host their LLMs to ensure better control, customization, and data security. This introduced challenges related to infrastructure, maintenance, and seamless integration with the game's existing systems. Patches: MMORPGs are dynamic, with new content, quests, and lore frequently added. The Oracle NPC had to be designed to quickly assimilate this further information, ensuring its knowledge base remained up-to-date and relevant without requiring extensive manual intervention. Solution The system was primarily built using Rust to handle the immense scale and volume of simultaneous player queries. This programming language ensured speed, scale, and thread safety, offering a significantly more efficient performance than traditional Python-based solutions. Rust's memory safety features ensured the system remained stable despite heavy loads. The latency challenge was met head-on by leveraging GGML for efficient LLM inference and quantization. This ensured Oracle's responses were accurate and delivered with imperceptible delay, maintaining the game's immersive experience. To make Oracle's knowledge adaptive and tailored to each player's progress and to have up-to-date info on new patches, we've built several data pipelines to transform and feed the data into Milvus and FAISS. These technologies facilitated efficient knowledge storage and retrieval, allowing the system to quickly access relevant information based on a player's achievements and game status. Also, additional metadata-based tweaks were done to efficiently answer SQL-like questions. The decision to host their LLMs introduced infrastructure challenges. However, we've achieved dynamic load balancing and scaling by utilizing Kubernetes (EKS) with AWS. This ensured the system could handle varying loads, from quiet times to peak player activity, without any hitches. Fine-tuning LLaMa 2 with Python & PyTorch: The Oracle's unique voice and integration of game lore were achieved by fine-tuning LLaMa 2 using the HuggingFace toolset powered by PyTorch. The instruct dataset gave the Oracle a specific style, seamlessly weaving the game's lore into the LLaMa 2 weights. Impact The introduction of the Oracle NPC transformed the MMORPG's gameplay dynamics. Players now had a unique, interactive knowledge base, deepening their engagement and immersion in the game's expansive world. With Oracle's ability to provide tailored information based on individual player progress, gamers experienced a personalized journey. This dynamic adaptation enriched the gameplay and reduced the need for external game guides or forums, keeping players within the game environment. The robust technological backbone ensured that Oracle NPC could cater to many players simultaneously without latency issues. Read More

Knowledge & Memories Agent for Bookstore Client Services: AI Solutions Development Summary The client operates a bookstore with various titles spanning genres, authors, and eras. This store is not just a commercial entity but a hub for literary enthusiasts, providing curated collections, rare editions, and a space for readers to delve deep into the world of literature. The client envisioned a Conversational Agent that would serve as a knowledgeable companion for customers. This agent would possess in-depth knowledge of the content of books on sale and a vast understanding of classic literature. Beyond just answering queries, the agent should recommend books based on user questions. Furthermore, it would have a 'memory' of past interactions, allowing it to offer personalized recommendations based on previous conversations with the user. A system was built using entirely API-based LLMs. The main LLMs for generation and embeddings were GPT-3.5 and ada-002. We used Pinecone for knowledge & memory storage. This combination allowed the Conversational Agent to provide insightful book recommendations and maintain interaction continuity. Tech Stack Python LangChain Pinecone OpenAI API AWS Lambda TECH CHALLENGE Books, by their very nature, contain layered and multifaceted information. The challenge was to design a system that could understand and navigate this hierarchy, from high-level themes and plot summaries to intricate details like character motivations or specific events. The agent should be able to delve into any layer of a book's content based on the user's query. Beyond just knowing the events of a book, the agent has to comprehend the personalities of the main characters. This means understanding their motivations, relationships, growth arcs, and how they react in various situations. Deep, nuanced knowledge is essential for answering questions about character traits or predicting hypothetical scenarios. The agent is expected to understand individual books and draw comparisons between them. Whether grouping books by similar themes, comparing character arcs across different novels, or ranking events based on their significance, the system has to be adept at comparative literary analysis. Users might ask the agent to rank books or characters based on criteria such as moral complexity, romance, or suspense. This required the agent's flexible understanding, allowing it to rank concepts based on varying user-defined criteria dynamically. One of the most challenging aspects is ensuring the agent remembers past interactions. This 'memory' would allow it to provide contextually relevant recommendations, building on previous conversations. Implementing such continuity in a conversational agent, especially dealing with vast literary data, is a significant technical hurdle. SOLUTION Addressing the complex hierarchical knowledge challenge began with LangChain, a framework designed to prototype conversational agents rapidly. This allowed the team to quickly iterate and refine the agent's capabilities, ensuring it could navigate the intricate layers of book content, from overarching themes to minute details. We've used a couple of "map-reduce" techniques. Reduce being summarization that was relatively quickly adapted and iterated over with LangChain. Similarly, past conversation history is transformed into memories. GPT-3.5 powered the core of the Conversational Agent's knowledge and response generation. Its expansive knowledge base was crucial for character personality derivation and comparative analysis. Meanwhile, the ada-002 algorithm created embeddings, enabling the agent to understand, compare, group, and dynamically rank literary concepts based on user-defined criteria. Pinecone was employed to manage the vast literary data and ensure the agent could quickly and accurately retrieve relevant information. This vector database was instrumental in storing the hierarchical knowledge, providing the agent could seamlessly delve into any depth of a book's content. Since all the processing is purely API based, we could leverage serverless architecture due to having any state within the code and no "heavy" computations. So we've leveraged AWS serverless architecture toolbox to optimize the cloud cost, have 100% usage-based charging, and be able to scale 1000x at any load spike. IMPACT The implementation of the Conversational Agent transformed the bookstore's customer engagement. Readers now had an intelligent literary companion, guiding them through the vast world of books with personalized recommendations and deep insights. With the agent's ability to remember past interactions and provide tailored book suggestions, customers felt a deeper connection to the store. This personalized touch increased sales and ensured customers returned, eager to continue their literary journey with the agent's guidance. Integrating an advanced AI-driven solution solidified the bookstore's reputation as a forward-thinking and innovative establishment. This attracted tech-savvy readers and set the store apart in a competitive market, drawing the attention of literary enthusiasts and technology lovers.  Read More

Virtual Wine Consultant for a Wine Company Client Services: AI Solutions Development Summary The client is a global wine discovery and marketplace platform. They created a membership-based wine community to unite wine enthusiasts, share knowledge, and assist in creating unique wine collections. The client decided to develop a Conversational Agent who would assist wine collectors by answering questions about different wines in their storage (cellar) and their characteristics. Additionally, the Agent can answer general questions about the wine industry and all wine varieties. The final solution would be feasible in Metaverse, with a virtual consultant available through the company's VR add-on. A system has been developed leveraging API-based LLMs. OpenAI GPT-3.5 and a set of customized models for question classification, embedding creation, and named entity extraction power the solution. Milvus served as the vector database for knowledge storage. This tech stack combination built up the search engine and made the agent interactive. Tech Stack Python AWS cloud stack Deberta LM Spacy NER Milvus Postgres OpenAI API  Tech Challenge Wine knowledge is a complex science, with each bottle presenting unique characteristics. The Conversational Agent should precisely navigate the database from each wine cellar, answering questions from wine types and flavor profiles to the geography of vineyards. It should analyze such factors as grape varietals, terroir influences, and aging processes. As a result, the virtual wine consultant would provide qualified replies and personalized recommendations. Furthermore, the Agent should be capable of drawing parallels between different wines. Whether comparing wines based on similar flavor profiles, terroir characteristics, or aging process, the system needs to excel in comparative wine analysis. It should give dynamic and engaged replies, as a real wine consultant would do. The major challenge was to find a way to match the user question represented within the natural language with a structured query in the cellar database and answer it in a human-like format after extracting information. Solution The system must handle 2 main cases of user questions: questions about wines that the user already has in his cellar (specific details about the chosen bottle, information about the quantity or presence of the selected wine, listing wines according to given filters like color, region, classification, etc.) and questions related to the wine topic in general. To understand the user's intent, we classify the question with Deberta on the dataset created by our team. Further, we augmented it using ChatGPT to reach the data volume enough to fine-tune the selected model. To handle generic questions about wines, we have created a vast knowledge base containing domain information. To implement it, we processed different wine-related books to create a database of short topic-specific textual pieces of information. All this processed information is uploaded to the Milvus vector database, which allows users to search for relevant information by question. The vector search is based on embeddings produced by the gte-large model. After the most relevant pieces of information are found, they are passed into GPT-3.5 Turbo to summarize it and generate a human-like response for the user. To handle questions about wines that the user already has in his cellar, we created a wide range of templates used to map user questions into predefined SQL queries to select the necessary information from the structured cellar data. All the cellar information is stored under the Postgres database. To efficiently map the user question into a relevant template, we trained the Spacy Roberta-based NER model to extract specific wine-related entities (such as color, region, classification, vintage, vineyard, etc.). This allows us to reduce the search space for the relevant templates. When the most relevant templates are found, the corresponding queries are executed, and the information from the Postgres database is passed into GPT-3.5 Turbo to summarize it and generate a human-like response for the user. Operating solely via APIs enabled the adoption of a serverless architecture, leveraging the AWS toolbox to optimize cloud costs and seamless scalability during load spikes, all under a 100% usage-based charging model. Impact The virtual wine consultant became a powerful community tool. It helps wine collectors receive up-to-date information about their wine storage and quickly access all the wine-related information to broaden their wine knowledge. Given the System's wide knowledge and ability to extract relevant information and the final Metaverse wrap-up, the wine consultant can establish connections with the user, analyze requests, and demonstrate high levels of intelligence and responsiveness. The solution is a part of the client's strategy to bring the wine industry to the new digital and commercial frontiers. Read More