In this article, you'll learn:
Challenges related to AI adoption faced by companies
How Portkey.ai addresses these issues with its observability suite and AI gateway
Key insights on fundraising and investor support in the AI space
Hard lessons learned and practical advice for AI startup founders
Insights on market focus and global expansion
Real-world AI Challenges
Genesis:
At Freshworks, Rohit Agarwal and his team embarked on an ambitious project to integrate GPT-2 models into their customer support framework. However, they quickly collided with the harsh reality of emerging technology—production accuracy was frustratingly low, severely hindering effective implementation. This challenge underscored the complexity and infancy of practical AI applications at the time.
As technology progressed, the release of GPT-3 offered a glimmer of hope with its enhanced capabilities. Yet, this advancement was not without its own set of challenges, particularly when Rohit moved to Pepper Content. Here, the stakes were higher, with a staggering 250,000 users engaging with AI-powered content generation. The team grappled with new hurdles:
Managing Massive Scale: Ensuring smooth interactions for a quarter of a million users was no small feat.
Navigating API Limitations: Frequent rate limits on API calls became a critical bottleneck, stifling user experiences.
Deciphering AI Behavior: The AI models were like black boxes—powerful yet often opaque, making debugging a daunting task.
Ensuring Consistency: The performance of AI applications was erratic, with unpredictability that could make or break the user's trust.
Despite these obstacles, Rohit’s team rapidly developed a proof of concept within just two weeks. However, the initial success was just the beginning. Over the next two years, they dedicated themselves to crafting sophisticated tools designed to tame the complexities of AI, enabling scalable and efficient implementations.
Portkey.ai's Solution
Portkey.ai addresses these issues with two key components:
a) Observability Suite:
Logs and analyzes AI deployments
Tracks 40+ LLM metrics for price, performance, and accuracy
b) AI Gateway:
Acts as middleware between AI apps and LLMs
Provides a unified API for various LLM providers
Offers load balancing, fallback, and retries
Additional features include governance (virtual keys, rate limits), control (traffic management, prompt versioning), and safety measures against malicious content.
Fundraising Insights and Investor Support
Starting Portkey.ai in April 2023, the team faced a constant dilemma of market timing - whether they were too early or too late to the market. This uncertainty influenced key decisions on product development pace and market direction.
The approach to resolving these issues included:
a) Strategic Angel Investors:
Brought on board angels from Databricks, OpenAI, and Cloudflare
Included the CRO of Asana for Go-To-Market insights
b) Venture Capital Support:
Secured backing from Lightspeed and Peercheque
c) Continuous Market Validation:
Regularly talked to customers and industry experts
While expert advice was crucial, many decisions came down to educated guesses in a fast-moving market. The key was having a support network to help navigate these uncertainties.
Hard Lessons and some insights for AI Startup Founders
Building Portkey.ai provided valuable insights for founders in the AI space:
a) Keeping up with AI advancements:
The rapid pace of AI development creates a constant fear of missing out (FOMO)
Focus on identifying problems that GenAI can solve exceptionally well
Avoid the "hammer looking for a nail" approach; instead, find use cases where AI can quadruple productivity
b) Understanding GenAI limitations:
Recognize that GenAI is primarily a text generation tool
Be aware of its limitations in areas like mathematical calculations
c) Navigating market timing:
Continuous questioning of whether the product is too early or late for the market
Balancing the dilemma of accelerating or slowing down product development
Confidence in market direction based on experience, while acknowledging the fast-moving nature of the AI field
d) Leveraging investor and advisor support:
Strategic angels provided crucial insights where Portkey.ai felt they needed some help
Venture capital support helped validate market assumptions
Emphasize the importance of talking to customers and industry experts for course correction
e) Global expansion considerations:
Setting up a US entity for fundraising and sales requires careful planning
Consult with a good lawyer and learn from experienced founder friends
Consider the pros and cons of relocating or managing remotely
Market Focus and Global Perspectives
Several factors drove the decision to focus on the US market:
The US AI market is more mature, with products already in production
SaaS products generally have a larger market in the US
India's market still values services more than software products
The overall market size for AI tools is currently larger in the US
However, the Indian market is expected to catch up over time as AI adoption increases and the ecosystem matures.
Key Takeaways:
AI implementation at scale presents unique challenges that require specialized tools and approaches
Success in the AI startup space requires balancing technical knowledge, market understanding, and strategic partnerships
Continuous market validation and adaptability are essential for navigating the uncertainties of the AI startup landscape
Understanding the limitations of GenAI and focusing on high-impact use cases is crucial for product development
The choice of market focus should consider factors like AI adoption rates, market size, and the balance between service and product preferences
Building a strong network of advisors, investors, and fellow founders can provide valuable insights and support in the fast-moving AI space