Introduction
As the transformative power of Generative AI and large language models sweeps through corporate processes, new challenges knock at our doors. How can these models interact seamlessly with internal ERPs and CRMs? Can they handle sensitive customer data without compromising privacy and security?
Recent advancements have seen open-source models significantly improve in performance, with autonomous agents like AutoGPT, built on these large language models, showcasing the ability to solve intricate tasks while integrating smoothly with external systems. But to harness such cutting-edge innovation, we must rethink and reshape the existing IT infrastructure, while incorporating new processes and paradigms. This article envisions a future where open-source and cloud-hosted models co-exist as equals in the corporate Generative AI stack.
Navigating the Hybrid Model
While tech giants like Microsoft Azure and Google are pioneering solutions for interacting with their large language models without transferring all content, these approaches come with their caveats – they’re costly, need fine-tuning with existing data, and require an unwavering trust in these providers’ processes.
Open-source models, on the other hand, can be run within an organization’s infrastructure. Their performance has skyrocketed, making them capable of executing tasks usually run on publicly hosted alternatives without posing data privacy and security risks.
The choice between a self-hosted, possibly fine-tuned open-source model, and a public model should be dictated by the use case. Especially when dealing with critical internal systems like ERPs and CRMs, the additional control over the model and the data that self-hosting offers can be invaluable.
Another advantage of text interface models is their interchangeability. Except for scenarios involving embeddings and vector search where switching might require generating new embeddings, enterprises can continuously benchmark different models and update them as newer ones become available.
Selecting the Right Model: Factors to Consider
When choosing the appropriate model for a task, several factors come into play:
- Performance: Which model excels at a particular task? Is fine-tuning needed? Depending on data availability, both options may have different benefits.
- Data Privacy: For interactions involving personal information or trade secrets, self-hosted models can be treated as internal systems, eliminating additional risks.
- Interaction with Internal Systems: Self-hosted models, by virtue of running within the corporate infrastructure, can integrate with existing authentication and authorization schemes, and eliminate the risk of critical information leakage.
- Cost: Depending on the use-case and the volume of data processed, either variant can be more cost-effective.
Consider these examples:
- Agents generating reports based on public information need only access internet sources and can freely run in the cloud without restrictions.
- Conversely, automating customer service, which involves interacting with the CRM, internal databases, and processing sensitive customer information, is better suited to a self-hosted model fine-tuned on existing customer conversations.
Let’s consider two examples:
Example 1: Agents to Create Reports Based on Public Information
Imagine an enterprise that relies heavily on data gathered from public sources to generate industry reports. Such a task might involve pulling information from various online news outlets, government databases, or social media trends. The scale of this operation, combined with its public nature, makes it an ideal use case for a cloud-hosted generative AI.
A cloud-hosted model, trained on a vast expanse of the internet, excels in scouring the web and analyzing complex, evolving patterns. These models can extract and process the necessary information quickly and efficiently, thanks to their robust computational power and unparalleled breadth of training data. However, the agents don’t interact with any proprietary or sensitive data, negating the need for high-security measures. Therefore, running these tasks in the cloud offers both practical and economic advantages without any compromise on data privacy or security.
Example 2: Automating Customer Service
Contrast the previous example with automating customer service, a task that’s at the other end of the privacy spectrum. Customer service involves handling sensitive customer information, interacting with internal databases, and often dealing with complex, personalized queries. Here, a self-hosted, fine-tuned model emerges as the preferable option.
A self-hosted model allows for the highest level of control and security, treating customer data with the confidentiality it deserves. This model can be fine-tuned on existing customer interactions, ensuring it’s not just intelligent but also empathetic and aligned with the company’s voice and values. It seamlessly integrates with internal systems like CRMs and proprietary databases, providing quick access to relevant customer data. All interactions occur within the secure confines of the company’s infrastructure, eliminating any data leak risks. Furthermore, since the model is self-hosted, the enterprise has complete control over updates and improvements, ensuring the AI continues to evolve alongside the business and customer expectations.
Avoiding Early Vendor Lock-ins
From the inception of your generative AI projects, consider a hybrid model approach. Frameworks like LangChain are effective tools that simplify the interaction with a model provider.
Establish a model and provider selection process ensuring that every project evaluates the relevant factors to choose the right model and provider.
It’s crucial to cultivate the competence to run, maintain, and fine-tune open-source generative AI models within your infrastructure securely and at scale. Don’t let the glitz and speed of innovation blind you to the importance of making sound, long-term strategic decisions. Today’s MVPs can quickly become tomorrow’s legacy software. Integrating generative AI into the corporate world is a marathon, not a sprint.
Take the time to establish a generative AI strategy, infrastructure, and guidelines. Inform, train, and empower your staff, particularly software engineers and product managers.
Conclusion
We are on the precipice of an exciting new era filled with pioneering technologies and opportunities. Yet, the future is not without challenges. To promote widespread adoption, we must conquer these hurdles.
Building a robust internal infrastructure that’s compatible with major players like OpenAI, Azure, and Google is advantageous compared to risking vendor lock-ins. Such a foundation allows you to leverage the best of both worlds – the innovation of cloud-based solutions and the control of self-hosted models – and paves the way for a seamless generative AI transition.
