In this post (Day 5 of 20 posts in our series on Generative AI), we'll discuss the emerging narrative around 'Copilot for X', where X is a typical knowledge task in a typical business setting.

This has become the dominant UX for both:

• Incumbents (enterprise apps deploying AI) or
• Startups (AI apps automating enterprise processes)

But has someone articulated what does being a copilot for knowledge work mean?

What is an AI copilot for knowledge work?

Most professionals hire junior, less experienced humans to help them tackle the routine, repeatable, (some may say) process oriented portions of their job - so that they can focus on the really complex parts of the problem, which perhaps needs tonnes of context, experience and cross-functional skills.

Think of AI copilots as your digital knowledge assistants focused on a sliver of the business process. Knowledge work usually comprises of large volumes of either text or numbers or both - something that latest AI models specialize in.

What are the key characteristics of a good AI copilot?

1. Are present where you work: Copilots have to be seamlessly integrated inside your primary work tool (word, excel, browser, messenger, trading platform, etc.) so that you are not wasting time alt-tabbing screens to access AI. Expecting users to learn a new piece of software (i.e. your fancy app) to leverage AI is just adding one layer of friction.
2. Enable smart, interactive collaboration: Copilots designers need to be deep domain experts so that dominant patterns of your typical tasks (JBTDs) become standard options. It should offer 2-3x efficiency to begin with‍
3. Have low latency: Response from AI models have to be closer to your speed of analysis or typing. Otherwise, you will have additional coginitive load of running two parallel threads in your mind - one for the actual task and other to keep track of the AI)‍
4. Detect user's intent: This one is tough - a well designed copilot should be able to decipher what are you likely to do next in a series of steps around a task, and present those options / ideas, rather than waiting for you to click buttons. Most copilots in the industry have not achieved this UX.
5. Does not only / always mean a chatbot: Having a conversational interface is nice, but that does not suffice for a copilot.
6. Can access and use your knowledge repository: A well designed knowledge assistant should be able to speed up work just because they have access and tremendous search / retrieval capabilities over your knowledge base. It won’t make your coffee, but it’ll brew up answers faster than you can say "Elementary, my dear Watson!" Just remember, it's above average but not infallible – the real detective work is still up to you!

Why is 'Copilot' a good approach to deploy AI in knowledge tasks?

We'll build the argument in 3 parts:

1. Cost of Error vs Value of Efficiency
2. Copilot vs Autopilot
3. What about adoption?

Cost of Error vs Value of Efficiency

Lets take a short detour towards the aviation industry - which has a long history of incremental technological advancements. In flight navigation, 'Cost of Error' cannot be any higher (its literally a matter of life and death). Spurred by tremendous investments by OEM's and managed under an extremely robust governance / oversight framework, Autopilot systems, which date back to the early 20th century, have evolved from simple mechanical devices to complex, computer-driven systems that can handle many aspects of flight. This transition didn’t replace pilots but transformed their role, emphasizing monitoring and decision-making over manual control.

Now, coming to knowledge work, lets take a look at few factors:

• Demand - given the digitization of our economy and society, there is an insatiable and ever increasing demand for professionals who can perform 'knowledge work'. Trend of knowledge workers far outnumbering (and out-earning on an average) factory works has continued for last few decades and looks to become stronger. Hence, the business case for more through-put (i.e. higher efficiency) is strong
• Cost of Error - while there are significant financial, repetitional and operational risks in errors committed in knowledge work, it hardly compares to aviation. This is not to assert that knowledge work can be full or errors or oversights, but just the fact that it is relatively easy to react to an initial draft & fix its issues (generally) than create new work (having some marble to carve). A well-designed UX, driven by an AI Copilot reduces chances of a severe error in highly knowledge intensive tasks.‍
• Accuracy / Learning Rate - most knowledge tasks produce tremendous amounts of 'data exhaust'. That enables LLMs to learn on bigger datasets each time and coupled with improvements in algorithms, the accuracy of LLMs has been increasing at a non-linear rate in last 3-4 years‍
• Human in the Loop - a copilot paradigm not only enables the human to be the review layer but also enables humans to retain control of the final output. This is an important psychological aspect of knowledge work.
• Embedded in your favorite workspace - a copilot is designed to cause minimal disruption to your workflow, requiring minimal cognitive load required  to learn a new system. Hence, its usually embedded inside your product / tool of choice. (Word for contracts ;) )

We believe these factors combine to makes Copilot the right paradigm for infusing AI into knowledge work

‍

Here is what we think about copilot for contracts:

Copilot vs Auto-pilot

‍

‍

We believe that today's models represents strong, narrow form of AI, and the best opportunity to integrate AI is with human in the loop, i.e. in a copilot paradigm.

One of the best examples of companies attempting Autopilot is Tesla FSD. And while the AI is stunningly advanced and improving every day, the engineers have realized that achieving the last 1% in accuracy is 100x more difficult than the achieving the last 10% in accuracy, etc. So, it might be counterproductive to wait for the models to become sufficiently advanced that autopilot capabilities could be achieved.

As a business manager, the value that you can extract from Copilot exists today, not in discounted future.

One can also categorize AI products based on user experience. Lets define these two axes:

1. Level of user input and control - this determines how much does the product rely on implicit data analysis or user behavior to function. In some cases, users provide some level of input or control AI's behavior through prompts, settings or feedback mechanisms. In other cases, users will steer and control AI's behavior significantly through extensive prompting, parameterization and appropriate control mechanisms.
   
2. User trust and confidence - this is a critical aspect in adoption as there's widespread concerns about bias, security or ethical implications. Users may have some level of trust in the AI but might require reassurance or evidence of its accuracy and reliability.

We have put some widely known AI products on these two axes for clarity:

Addendum: Here is a mainstream media coverage of an AI Autopilot deployment claim by a big contract AI provider: https://www.cnbc.com/2023/11/07/ai-negotiates-legal-contract-without-humans-involved-for-first-time.html

We believe that current generation of AI models are far from supporting an 'Autopilot' capabilities, if they were ever required in contract drafting and negotiations. While this article is a word salad of AI buzzword and contract review terms, there's no description of why cutting a human out of such a critical process is beneficial. These sort of half baked media briefings do more harm than good to the adoption of AI by legal industry.

Adoption of AI copilots in knowledge work

Since the copilot paradigm originated (mostly) with 'Github copilot', lets uncover how a 'coding assistant' can help coders across various levels of proficiency. By their own definition, Github copilot is an 'AI-pair programmer that can help developers write code more efficiently'. It can

• Save time: By generating suggestions for entire lines of code
• Improve code quality: By catching potential errors early
• Reduce errors: By automating repetitive coding tasks
• Increase productivity: By streamlining the development process

‍

‍

We believe that AI copilots are still in their infancy. As of late Oct '23, Github Copilot claims to have over a million paid users (that's about 1% of total Github's user base), across 37K organizations. By any stretch of imagination, that has to be the largest ever deployment of an AI product. There are known issues like it producing code that is not functional or the drift between intent and execution, or code that does not answer the intended problem. Other issues exist like: 

• It heavily indexes on Python has has a poor coverage of other programming languages,
• Potential security flaws,
• And lack of integration with other tools.

Copilot products from majors in the market

Microsoft

Google

Github

Perplexity

Add your favorite copilot product in comments.

‍

We believe that Microsoft's recent launch of Copilot in Office Productivity Suite will further cement this idea. Adoption is likely to be strong due to:

• LLMs continue to make regular and significant gains.
• Incentives are aligned for users/buyers, application / AI copilot creators, and investment managers.
• No fresh transformative research required because the technological components are already in place.
• A pattern of steadily shorter times for embracing new tech platforms.

‍

‍

In conclusion, we believe that the constraint on changes is not technological. From an R&D perspective, every technological component required to produce these game-changing goods is currently accessible. It is now necessary to assemble the parts. This new class of products offers businesses and knowledge workers a once-in-a-lifetime opportunity.

We may have just seen first few waves of the Tsunami caused by energy event that occurred on Nov 30th, 2022.