How we run our in-house generative AI accelerator: Framework for ideation

  • 11/17/2023 - 16:35
  • 1 Wiev

At some point, every scaling business faces the decision: Do we stick to our core activity or venture into uncharted territory? It’s a choice between enhancing existing products and branching out to create something entirely new.

For tech giants willing to expand into uncharted territory, the common approach often involves acquiring early-stage companies or even teams and integrating them into their ecosystems. This is because for mature organizations, breaking away from their core activity is no easy feat. In practice, going from zero to one is always more difficult than scaling from one to 100. In fact, less than 8% of ventures launched internally reach scale.

And the mergers and acquisitions (M&A) strategy may not necessarily be the answer for growth either. This approach demands deep pockets, a luxury primarily enjoyed by industry titans such as Adobe, Google, or Meta, along with the expertise to acquire and effectively integrate ventures. And still, 70% to 90% of acquisitions end in failure.

At Reface, we embarked on a difficult challenge to learn how to build new businesses within the company. In this article, I’ll unveil our framework for evaluating new product ideas and how we determine whether a concept merits our pursuit.

Enlightenment or methodology?

Classic Y Combinator advice on “noticing” ideas is undoubtedly a great way to discover ideas; however, it cannot be the main approach for a company that needs to constantly generate ideas to work on. Therefore, through trial and error, external advice, and extensive research on the topic, we have been developing our own methodology. This methodology can result in a higher number of ideas that are good enough for testing compared to just following the “noticing” path.

The first essential step to cultivate innovation within an established company is to create a dedicated space where new ideas will be born. In the daily grind of a functioning business, employees are usually occupied with their daily tasks and tend to stick to established processes and clear objectives, rather than exploring new ideas or initiating new projects. Therefore, it becomes essential to establish conditions that encourage idea generation.

This can involve setting up a small, dedicated team (ideally no more than five people) responsible for managing the idea backlog, creating an analytical feed of ideas, and developing a framework for evaluating and prioritizing ideas based on your company’s strategy. I believe in collective brainstorming during the initial stages of idea exploration, gradually transitioning to clear ownership during the idea execution phase to closely resemble the circumstances of a founder launching a startup.

Boston, MA July 15

Building an analytical feed of ideas is equally important. We designed a system that parses recent ML research papers, reacts to trends in social media and search spikes, and tracks various other sources of inspiration.

For the entire ideation process, we offer two key pieces of advice. First, adopt a laser-focused approach with concentrated sprints aimed at a group of ideas. When one idea is rejected, swiftly move on to the next. Second, focus on identifying promising idea spaces, not a specific idea. While the initial idea may not succeed, exploring the right space can lead to the discovery of viable alternatives. In essence, we hunt for fertile ground, trusting that within it, the seeds of innovation will naturally sprout.

Finally, it’s crucial to have a framework for evaluating ideas. After a few weeks of idea generation, you will likely find yourself with numerous ideas, making it challenging to assess and prioritize them without a framework in front of you. Therefore, I will share the methodology that works for us in consistently creating, developing, and testing new product ideas inside the company.

Evaluating the idea Space: Four criteria to follow

flowchart showing framework for evaluating idea space in gen AI
Image Credits: Anton Volovyk

1. Problem: Is your product a painkiller or a vitamin?

Nothing has changed throughout centuries: Every product is designed to address a specific problem faced by its users. Therefore, it is essential to understand what hurts and how it can be cured. How essential is your product idea? Is it a painkiller or a vitamin? Assessing this isn’t as straightforward as it may seem. For example, Character AI might appear to be a one-time entertainment tool with no apparent problem-solving function. However, it boasts a remarkable retention rate, exceeding 30% after 30 days [Sensor Tower], a performance level akin to the best social products on the market. By September 2023, it had garnered 4.5 million active users in the U.S. This data serves as evidence that AI chatbots are addressing a substantial issue — the problem of loneliness and social isolation — that remains highly relevant for our generation.

When researching a problem, also consider:

  • Demand: Is there evident demand for a solution to the problem? Be it through online forums, social networks, or casual conversations with friends, any evidence of this demand reinforces the notion that the problem indeed exists, and people are willing to invest in its solution.
  • Existing solutions analysis: We meticulously evaluate the landscape of existing solutions for the given problem. In fact, we like to discover competition actively addressing a specific issue. If a successful business has been established to address a problem, it proves the existence of the problem and the user willingness to pay for its solution. However, we also scrutinize the quality of the solution. Is it tailored for a specific user base? Is it technologically advanced? Is it well-distributed? A prime example of a “strong idea space” is Excel, which has evolved for more specialized use cases. Numerous successful startups have emerged to supplant Excel, including CRM systems, HR software, personal finance management applications, and more. These tools all originate from the common spreadsheet, once used for organizing information, but have since matured into specialized applications with targeted functionality and design to address diverse user needs.
  • Frequency of use: Some products naturally have low usage frequency, like Airbnb, where users might engage only once every six months. While such products can be excellent, their infrequent use necessitates expandable strategies to keep users engaged and retained.
  • Market size: Every industry has its own fundamentals, but in the B2C apps market, a new product is unlikely to capture more than 10% of the market. Therefore, it’s crucial to focus on problems of sufficient magnitude where capturing a realistic market share aligns with your objectives.

2. Product hypothesis — either better or notably different

After analyzing the potential of an idea space, proceed to brainstorm and develop a product hypothesis. Try to answer two pivotal questions here. First, is your product superior (in some way) to existing alternatives? Second, can it generate revenue and build a profitable business model?

Consider a Delta-4 approach — assess existing products in the market and rate their performance on a scale of 1 to 10 in regard to different problems/audiences. Only enter the market if you can convincingly identify at least a four-point difference in quality, cost-effectiveness, or some other distinguishing factor. Even if you can create a product rated at a 10 but existing solutions already score a 7, the inertia of customers may prevent them from switching. Thus, when entering a market, strive to be either better or notably different, at least for some customers and use cases.

Furthermore, given that we are discussing an idea accelerator within an established company, it’s crucial to model required product and business metrics. Begin by outlining a hypothetical business model with specific objectives, and then work in reverse, running cheap tests to validate as many critical assumptions in the model as you can. This business model will also come in handy if you actually launch the product, and will serve as a benchmark for your actual metrics, offering tons of information for your team.

3. Why now?

Look for a market that has experienced, or is about to experience, a significant transformation — a wave, a storm, or a paradigm shift. This may be triggered by technological advancements, regulatory changes, or evolving customer habits. At Reface, we are particularly attentive to markets with technological changes, as they create opportunities for newcomers. A good example is the emergence of LLM capabilities and their potential impact on the gaming industry, which now allows for the creation of smart NPCs and an endless array of plot lines. We believe that with these profound changes, new entrants have significantly greater chances of success compared to a few years ago when innovation in the gaming industry was more incremental.

Furthermore, market growth is an undeniable indicator of opportunity. When the entire market is on the rise, every participant has the potential to thrive.

4. Cost of a prototype

Following all the previous steps will significantly increase your chances of setting up an effective ideation process inside the company. We have also discovered that a good ideation process is 4x to 8x cheaper than launching an MVP. However, you will never truly know if an idea will succeed until you launch it. Therefore, try to estimate the cost of an MVP and be strict in terms of both investments and the time you allocate to it. As a rule of thumb, only 1 out of 10 ideas will eventually work, so launch inexpensively and frequently.

I trust that our methodology for running our in-house gen AI accelerator can assist you in adopting a more systematic approach to the ideation process within your company. The framework I’ve shared has been particularly tailored to evaluate opportunities in gen AI but can be applied to other industries and businesses as well.

We would love to see more great ideas succeed in the coming years!

  • Etiketler:

Send a Comment

Information: Your e-mail address will not appear on the site.