Can a machine automatically generate a paragraph of fluent text? Not so long ago, the answer to this question was ‘No!’. Today, thanks to large language models, practitioners with different levels of expertise can build applications that generate useful text for marketing, content writing and more. 

Hold on tight, because in this blog post we’re going to go from an idea to a completely new text generation feature that you can ship immediately. Ready to get started? 

Motivation

Suppose you're building a job marketplace app or a personnel recruitment platform. You probably have a feature that lets candidates upload their CV to your system. Those CVs aren't always stellar. For example, many of them lack a profile section - a short paragraph that highlights the candidate’s background, skills and ambitions. Luckily, you can use AI21 Studio to help your users by auto-generating a CV profile for them. This is just one example of short-form content generation, where language models are used to draft text up to a paragraph long (more examples are in the summary).

In this post we'll build a CV profile generator that you can integrate into your application or website in just a few minutes of work.

Step 1: Define the problem

Understanding what kind of text you want to generate is essential. In this case we need to ask ourselves:  what makes a good CV profile that our users will appreciate? Let’s take a look at some examples and try to find ones we like. Here is an article you can find online that shows various types of CV profiles. For example, let’s look at a profile for a chemical engineer:

I am a dedicated, hardworking and proactive Chemical Engineer with a strong background in design, plant operations, offshore operations, and process and safety improvements. I have solid work experience in designing, testing and analysing processes to increase the overall efficiency of operations. I am currently looking for an opportunity to utilise my technical skills in a challenging working environment and become a valuable asset to the organization that I work for.

As you can see, it’s a short paragraph that covers the candidates background, experience and the type of role they are looking for. Browsing through more examples, it is clear that most CV profiles follow this pattern. Let’s make a note of a few examples (3-5 will do) that we like. Our goal is to generate texts similar to these examples. Now we are ready to log into the AI21 Studio and start building.

Step 2: Create your (initial) prompt

Language models take in some text as input and generate some likely continuation of that text. We call the input text the prompt and the generated text the completion. You can think of large language models as smart auto-completion algorithms: they are very good at latching on to hints and patterns expressed in plain English, and generating text that follows the same patterns. 

For example, if we input a prompt that is likely to be followed by a CV profile, we can get the model to generate new and original CV profiles as completions. One simple way to do this is to spell out an instruction, just like you would when asking a person to produce some text:

Write a winning Teaching CV profile:

Let’s take this prompt and see what the model will generate. Go to the AI21 Studio, put this prompt in the input field and click “generated”. This is the completion the model generated:

Mr George Mwangi is a Teacher who has completed Diploma, Degree and Masters Degree in Education. He has vast teaching experience of 7 years. His expertise is in the field of Science, Biology, Chemistry and Physics. He is a hardworking, dedicated candidate with vast experience of teaching various levels, including University students.

What do we like about this completion?

• It matches the profession we provided (Teacher). 
• It contains relevant education and experience.
• It uses some good adjectives.

What do we not like about this completion?

• This paragraph is written in third person (“Mr Mwangi is…”), whereas our good CV profile examples are always written in the first person (“I am…”).
• The details the model generated are completely made up - they may or may not reflect our candidates' true skills.

How can we make the model produce the output we want?

You can try to phrase the instruction differently. Be descriptive and clear, and try a few different ones, like:

Following is an exceptional CV profile for a Teacher:

or

Here is an enthusiastic Teacher’s CV profile:

That said, you shouldn't spend too much time on choosing the perfect adjectives. 

The most powerful tool you have at your disposal is providing examples that demonstrate what you expect from the model. This is the most effective way to guide the generations according to your intentions, and if you have time to do only one thing - this is it.

In this case, we already collected some relevant examples in Step 1. All that’s left is to create the prompt from instruction-profile pairs. Below you can find a prompt we made by stringing together a sequence of examples we liked. The bold text is the completion our model generated.

Did we solve all the problems? Well, the profile is written in first person, just like we wanted. Unfortunately, it still contains made up details. We solve this in the following section.

There are a couple of important things to note here:

1. The instruction we chose was “Write a winning … CV profile:”. It’s good to be consistent and use the exact same phrasing throughout the examples in the prompt.
2. Each CV profile ends with a pair of hashes, which establishes a pattern for the model to follow. As a result, when the model produces a new CV profile in the completion, it generates ‘##’ after it. We can use this signal to detect that the entire profile has been written and terminate the generation using the stop sequence parameter.

Want to see the generation or check another profile? Try it yourself in our playground.

Step 3: Add personalization to your prompt

The prompt we have at this point produces well-formatted, impressive-sounding CV profiles for specific roles. You can try it in the playground and see that it’s pretty reliable. Each time we generate a new text, we get a random profile for a candidate for the desired role. 

Although the profiles fit the role, they may be inconsistent with the candidates' actual skills, experience and ambitions. You probably want to provide your users a way to guide the generated profile to stick to the facts that are true for them. One way to do it is to inject specific features that you want to be reflected in the profile into the instruction.

We took the prompt from the previous section and tweaked it to include “features”. We extracted them manually from the examples we gathered earlier. This is the completion generated by the model:

You can see that now we can direct the model to include specific skills and experience!

Want to see how your features turn into a profile? Try here.

When working with prompts like these, it is critical to maintain a strong correlation between the features and the generation. If an example's output mentions facts that totally contradict the input or are completely original and unrelated, the model will follow suit and have a tendency to make up things. We call this phenomenon “hallucinations”. On the other hand, if you omit certain features from an example's output, the model will imitate this and occasionally drop some of the provided features.

To understand that, let’s take a look at the following bad example (do NOT try this at home!):

A single example (Business manager) was fed into the model here, but there is some mismatch between some skills and the profile. The table below shows both phenomena in this example: What was wrong with the prompt, and how it affected the completion.

‍

Step 4: Explore the parameters

Another way to affect the completion is to adjust some of the generation parameters. There are quite a few parameters and you can learn about them here. For now, let's focus on one of the most significant parameters: Temperature. 

The temperature controls creativity. Increasing the temperature tends to result in more varied and creative completions, whereas decreasing it produces more stable and repetitive completions. With temperature set to zero, the model will always generate the most probable completion for a given prompt (which will be the same completion every time).

Below are examples of three different generations using the same prompt from the previous step: 

Want to see more results? Try it yourself in the playground!

Step 5: Integrate with your code

After you've finished building your prompt and you are happy with the results you see in the playground, you'd want to integrate some code to call the AI21 Studio API in your product. You can click the API button and copy the necessary code snippet.

The request includes your API key (which is linked to your account), certain parameters, and more importantly - the prompt. In production the prompt will need to end with a different input every time we generate a completion, corresponding to the role and the candidate features. We'll need to write a bit of code that injects this into our prompt. Here's how you would do it in Python:

def create_prompt(role, highlights):
    """
        Arguments:
        - role         : what is the profession of the candidate
        - highlights   : what are the skills and ambitions of the candidate

        Returns:
        - prompt       : a full prompt, including few shot examples
    """
    
    few_shot_prompt = #Paste here the few shot prompt, ends with the stop sequence#
    current_input = f"Write a winning {role} incorporating the following features:\n"
    skills_str = ""
    for i in range(len(highlights)):
        skills_str += f"{i + 1:d}. {highlights[i]}\n"
    skills_str += "\nProfile:"

    return few_shot_prompt + current_input + skills_str

Check out this notebook for more details.

Summary

In this post, we have explored the process of going from an idea to an implementation of a short-form text generation feature with AI21 Studio. Other examples of short-form text that you can generate with AI21 Studio are tweets, online ads, product descriptions and more.

Creating a good prompt is a process that involves some trial and error, but it doesn't require a PhD in AI. So don't give up if your first attempt fails to produce good results. Look for good examples and keep iterating!

‍