Back to Research

Jurassic-1: Technical Details & Evaluation

August 1, 2021

Jurassic-1 is a pair of auto-regressive language models recently released by AI21 Labs, consisting of J1-Jumbo, a 178B-parameter model, and J1-Large, a 7B-parameter model. We describe their architecture and training, and evaluate their performance relative to GPT-3. The evaluation is in terms of perplexity, as well as zero-shot and few-shot learning. To that end, we developed a zero-shot and few-shot test suite, which we made publicly available ( as a shared resource for the evaluation of mega language models.

read paper

More upcoming events