Build with Jamba-Instruct on Snowflake Cortex

June 3, 2024

Announcing AI21’s partnership with Snowflake Cortex, making AI21’s groundbreaking language models more widely accessible through Snowflake Cortex’s fully-managed service.

June 4, 2024

‍

As part of its ongoing partnership with Snowflake, AI21 is thrilled to announce the upcoming integration of its state-of-the-art large language models with the Snowflake Data Cloud, starting with its latest and most advanced Foundation Model, Jamba-Instruct.

‍

The model will soon be available in Snowflake Cortex, a fully-managed service that enables customers to analyze text data and build AI applications using a range of industry-leading LLMs, AI models, and vector search capabilities.

‍

The partnership between AI21 and Snowflake Cortex democratizes access to the cutting-edge technology, placing powerful GenAI use cases securely within reach for all Snowflake customers. As Baris Gultekin, Head of AI at Snowflake, shares: “With AI21's groundbreaking Jamba-Instruct offered in Cortex AI, our customers will now be able to seamlessly connect their data to build transformative GenAI applications with AI21's powerful models.

Snowflake Cortex

AI21 is pleased to partner with Snowflake Cortex due to their commitment to streamlining the path to GenAI application development and deployment for all of their customers—whether they are advanced developers working in Python or analysts using SQL.

‍

As a fully-managed service built into the Snowflake Data Cloud, Snowflake Cortex supports their customers with:

‍

Seamless deployment of LLMs: Snowflake Cortex is natively integrated with the Snowflake Data Cloud, providing a single platform to simplify the connection between your organizational data and GenAI models.

Secure data access: As Snowflake Cortex deploys within the secure perimeter of the Data Cloud, your data remains fully within Snowflake, and the LLM you deploy with Cortex benefits from Snowflake’s existing security, scalability, and governance capabilities.

Infrastructure management: Snowflake Cortex optimizes the complex GPU infrastructure needed to run and scale GenAI solutions.

This makes using Jamba-Instruct as simple as writing a basic LLM function directly within Snowflake Cortex’s environment. As a result, customers can confidently use our models to focus on what matters most to them: building and innovating.

Building with Jamba-Instruct on Snowflake Cortex

AI21’s debut model on Snowflake Cortex, Jamba-Instruct, is a Foundation Model built from groundbreaking hybrid architecture. As the world’s first production-grade model to combine the conventional Transformer architecture with novel SSM architecture, it harnesses the advantages of both to achieve best-in-class performance and quality.

Jamba-Instruct excels across several significant milestones:

‍

Longest context window in its size class

With a 256K context window, Jamba-Instruct effectively handles the kind of long context use cases that are so valuable to enterprises, outperforming competitor Mixtral 8x7B across long context use case benchmarks. Roughly equivalent to a 800 page novel, a 256K context window opens up new opportunities to build powerful GenAI workflows at unprecedented scales.

*Results on long-context QA benchmarks, conducted using the same method outlined in section 5.2.2 of our Jamba base model* *whitepaper.*

For example, a financial institution can enable automated term sheet generation across many call transcripts, thereby increasing productivity, reducing human error, and absorbing draining, repetitive tasks so employees can focus on more strategic projects. Jamba-Instruct can also be used to implement a customer-facing chatbot that, due to its long context window, can maintain extended conversations that remain coherent over time.

‍

Behind the scenes, the model can also support a robust and sophisticated RAG Engine mechanism, able to draw on and process more of the company’s organizational knowledge base at once so that answers are grounded and accurate. These are just a few examples of the ways Jamba-Instruct’s unprecedented context window length supports enterprises with the kinds of strategic use cases and innovative GenAI applications.

‍

Cost efficiency

‍Offered at a highly competitive price point and achieving impressive quality benchmarks across the board, Jamba-Instruct is the clear model of choice for companies looking to build reliable GenAI applications at scale without ballooning costs.

‍

By bringing Jamba-Instruct to Snowflake Cortex, AI21 is excited to make this cutting-edge and cost-efficient technology highly accessible to a wide range of innovators—no AI or ML engineering required. In partnering with Snowflake Cortex, we at AI21 further our commitments to widening access to the LLMs needed to power high-value GenAI enterprise use cases and removing common barriers to production at scale by bringing our technology to Snowflake’s community of analysts and developers.

‍

To learn more and be the first to access Jamba-Instruct on Snowflake Cortex, please contact us.

‍

Back to Blog

Announcements

Build with Jamba-Instruct on Snowflake Cortex

MRKL Whitepaper

Paper: Standing on the Shoulders of Giant Language Models

What is a MRKL system?

In August 2021 we released Jurassic-1, a 178B-parameter autoregressive language model. We’re thankful for the reception it got – over 10,000 developers signed up, and hundreds of commercial applications are in various stages of development. Mega models such as Jurassic-1, GPT-3 and others are indeed amazing, and open up exciting opportunities. But these models are also inherently limited. They can’t access your company database, don’t have access to current information (for example, latest COVID numbers or dollar-euro exchange rate), can’t reason (for example, their arithmetic capabilities don’t come close to that of an HP calculator from the 1970s), and are prohibitively expensive to update.
A MRKL system such as Jurassic-X enjoys all the advantages of mega language models, with none of these disadvantages. Here’s how it works.

Compositive multi-expert problem: the list of “Green energy companies” is routed to Wiki API, “last month” dates are extracted from the calendar and “share prices” from the database. The “largest increase“ is computed by the calculator and finally, the answer is formatted by the language model.

There are of course many details and challenges in making all this work - training the discrete experts, smoothing the interface between them and the neural network, routing among the different modules, and more. To get a deeper sense for MRKL systems, how they fit in the technology landscape, and some of the technical challenges in implementing them, see our MRKL paper. For a deeper technical look at how to handle one of the implementation challenges, namely avoiding model explosion, see our paper on leveraging frozen mega LMs.

A further look at the advantages of Jurassic-X

Even without diving into technical details, it’s easy to get a sense for the advantages of Jurassic-X. Here are some of the capabilities it offers, and how these can be used for practical applications.

Reading and updating your databases in free language
AI-assisted content generation on current affairs
Performing simple and complex math operations.
Decomposing multi-step problems
Access to continuously changing information (weather, currency exchange rates)
Transparency and trust

Reading and updating your database in free language

Language models are closed boxes which you can use, but not change. However, in many practical cases you would want to use the power of a language model to analyze information you possess - the supplies in your store, your company’s payroll, the grades in your school and more. Jurassic-X can connect to your databases so that you can ‘talk’ to your data to explore what you need- “Find the cheapest Shampoo that has a rosy smell”, “Which computing stock increased the most in the last week?” and more. Furthermore, our system also enables joining several databases, and has the ability to update your database using free language (see figure below).

Jurassic-X enables you to plug in YOUR company's database (inventories, salary sheets, etc.) and extract information using free language

AI-assisted text generation on current affairs

Language models can generate text, yet can not be used to create text on current affairs, because their vast knowledge (historic dates, world leaders and more) represents the world as it was when they were trained. This is clearly (and somewhat embarrassingly) demonstrated when three of the world’s leading language models (including our own Jurassic-1) still claim Donald Trump is the US president more than a year after Joe Biden was sworn into office.
Jurassic-X solves this problem by simply plugging into resources such as Wikidata, providing it with continuous access to up-to-date knowledge. This opens up a new avenue for AI-assisted text generation on current affairs.

Who is the president of the United States?

Donald Trump

GPT-3

Donald Trump

Jurassic-1

Donald Trump

Google

Joe Biden

Jurassic-X

Joe Biden is the
46th and current
president

Jurassic-X can assist in text generation on up-to-date events by combining a powerful language model with access to Wikidata

Performing math operations

A 6 year old child learns math from rules, not only by memorizing examples. In contrast, language models are designed to learn from examples, and consequently are able to solve very basic math like 1-, 2-, and possibly 3- digit addition, but struggle with anything more complex. With increased training time, better data and larger models, the performance will improve, but will not reach the robustness of an HP calculator from the 1970s. Jurassic-X takes a different approach and calls upon a calculator whenever a math problem is identified by the router. The problem can be phrased in natural language and is converted by the language model to the format required by the calculator (numbers and math operations). The computation is performed and the answer is converted back into free language.
Importantly (see example below) the process is made transparent to the user by revealing the computation performed, thus increasing the trust in the system. In contrast, language models provide answers which might seem reasonable, but are wrong, making them impractical to use.

The company had 655400 shares which they divided equally among 94 employees. How many did each employee get?

94 employees.

GPT-3

Each employee got 7000 stocks

Jurassic-1

1.5

Google

(No answer provided)

Jurassic-X

6972.3
X= 655400/94

Jurassic-X can answer non-trivial math operations which are phrased in natural language, made possible by the combination of a language model and a calculator

Compositionality

Solving simple questions might require multiple steps, for example - “Do more people live in Tel Aviv or in Berlin?” requires answering: i. What is the population of Tel-Aviv? ii. What is the population of Berlin? iii. Which is larger? This is a highly non-trivial process for a language model, and language models fail to answer this question (see example). Moreover, the user can’t know the process leading to the answers, hence is unable to trust them. Jurassic-X can decompose such problems into the basic questions, route each to the relevant expert, and put together an answer in free language. Importantly, Jurassic-X not only provides the correct answer but also displays the steps taken to reach it, increasing the trust in the system.

Do more people live in Tel Aviv or in Berlin?

Berlin

GPT-3

There are more people living in Tel Aviv than in Berlin.

Jurassic-1

Berlin and Tel Aviv are roughly the same size

Google

(First hit is a comparison between Tel Aviv and Berlin)

Jurassic-X

More people live in Berlin than in Tel-Aviv

[‘Return population of Tel Aviv’; Return population of Berlin’; Return which is bigger between #1 and #2’]
Step 1: Population of Tel Aviv. Result - 451523.
Step 1: Population of Berlin. Result - 3664088.
Step 3: Which is bigger, #1 or #2. Result - Berlin.

Jurassic-X breaks down compositional questions, answers the basic sub-questions, and puts together the answer. Importantly, this process is transparent to the user greatly increasing the trust in the system

Dynamic information (like weather and currency exchange rates)

Certain types of information change continuously - weather, currency exchange rates, share values and more. Such information will never be captured by language models, yet can easily be handled by Jurassic-X by integrating it with a reliable source of information. We performed a proof-of-concept on two such features - weather and currency exchange rates, and the design enables quick integration with more sources to solve your use-case.
Weather - a loosely phrased question about the weather elicits an answer from all language models, where language models always return the same answer, regardless of when the question was asked (funny, right?), while Jurassic-X provides an answer based on the actual weather prediction.

I’m going to be in New-York in 3 days. Should I pack my umbrella?

GPT-3

Yes, you should pack your umbrella.

Jurassic-1

Yes, you should. The weather forecast is rain.

Google

(Links to weather websites)

Jurassic-X

Yes, you should pack your umbrella, because in New York in 3 days there will be broken clouds and the temperature will be -2 degrees.

Currency exchange rates change much faster than weather predictions, yet the Jurassic-X concept - a language model connected to a reliable source of information - easily solves this problem as well.

How much Moroccan money will I get for 100 bucks?

125 dirhams

GPT-3

100 moroccan dirhams is about 27$.

Jurassic-1

How much is 100 dollars in moroccan money?

Google

100 dirhams = 10.75 dollars

Jurassic-X

100 USD = 934.003 MAD

Jurassic-X combines a language model with access to APIs with continuously changing information. This is demonstrated for weather forecasts and currency exchange rates, and can easily be extended to other information sources

Transparency and trust

Transparency is a critical element that is lacking in language models, preventing a much wider adoption of these models. This lack of transparency is demonstrated by the answers to the question - “Was Clinton ever elected as president of the United States?”. The answer, of course, depends on which Clinton you have in mind, which is only made clear by Jurassic-X that has a component for disambiguation. More examples of Jurassic-X’s transparency were demonstrated above - displaying the math operation performed to the user, and the answer to the simple sub-questions in the multi-step setting.

Was Clinton ever elected president of the United States?

Yes

GPT-3

No, Clinton was never elected as president of the United States.

Jurassic-1

Google

Clinton was elected president in the 1992 presidential elections…

Jurassic-X

Bill Clinton was elected president.

Jurassic-X is designed to be more transparent by displaying which expert answered which part of the question, and by presenting the intermediate steps taken and not just the black-box response