How Harambee built conversational flows to achieve a 20% sign-up increase
Learn how Harambee used AI21 Labs' custom models to create an NLP chatbot app to onboard job seekers to one of their career programs.
Harambee is the official platform for government bodies wanting to hire youth in South Africa. But when the pandemic hit, they needed to find a low-touch yet incredibly authentic way to connect with the growing number of unemployed youth who were looking to get trained and hired.
So, how was it done? Harambee not only created a new virtual onboarding process, but also drove a 20% conversion rate increase in sign-ups by using AI21 as their Natural Language Processing (NLP) engine. This was achieved by setting up our Language Models to act as intelligent chatbots. The bots were able to understand the context of each conversation, build rapport with the individuals, and respond naturally – guiding them toward their career goals.
We spoke with Brent Davidoff, who leads Harambee's effort to scale transformative interventions using their omni channel contact center.
Who is Harambee?
Harambee Youth Employment Accelerator is a not-for-profit social enterprise that solves youth employment through partnerships. Harambee works with partners who are also committed to results that can work at scale – including government, the private sector, civil society, and over three million youth. By fusing brilliant minds with best-in-class technology, Harambee is working to unlock jobs and break down the barriers that keep millions of young South Africans unemployed.
But how they bridge the gap between unemployed with no experience to ready to work and employable is a key reason why Harambee eventually needed AI21 Studio.
Harambee put these youth through a simulation of a workplace that they call the “bridging process”. In it, the individuals report to a manager and are measured on punctuality and soft skills with a scorecard.
“After three months of daily bridging, they were transformed,” said Brent, noting that employers would always see a positive difference in these young adults after completing the program.
And then the 2020 pandemic happened.
Pivoting their processes, and turning to NLP during a pandemic
The once highly personalized, streamlined work socialization experience was now at risk. “With only 120 personnel at the time, we stopped doing face-to-face processes and didn’t do the bridging anymore,” says Brent.
Like many companies during that time, they went fully remote and launched a new website – SAYouth. And while they still had companies wanting to employ these youths, these individuals weren’t yet ready for the job market. “They were missing the secret sauce [of the bridging process],” adds Brent.
Searching for a Solution that Could Match Human Intelligence
This led them to the development of a chatbot that they called “Coachmee”. It created a conversation based learning experience that used evidence-based techniques to effect measurable behavior change in its users. But not without some snags along the way.
“From our previous work with other chatbots [before finding AI21], we had seen how the signup process was a vital component for ensuring high levels of engagement and most chatbots failed to do that with an audience like ours for a number of reasons,” Brent tells us, explaining why he went searching for a new solution.
Here’s what Harambee needed – and what led them to AI21.
Problem #1: A chatbot that can understand, not just repeat
Their focus was deeper than simply having quick response times for their customers. Harambee needed help creating trust, rapport, and relationships with their users during their sign-up process – a critical, pivotal moment in their program.
“Traditional intent-based NLPs are all about understanding what someone says, but then returning something that is prewritten. But no conversation is like that. You must be unique,” says Brent, providing us with an example of an everyday conversation during the sign-up process.
Chatbot: “Welcome to Coachmee, please tell me your name.”
Chatbot: “That looks like an ID number, would you mind, with respect, telling me your name again?”
Person: “My name is Rejoice Precious Mikateko ”
Chatbot: “Hello, Rejoice!”
“Already there, the ability to see someone and get their name right. “There’s no traditional intent-based NLP that can do that.”
Problem #2: Adapt to overcome language barriers and context
There are many models that offer a multitude of language options, but one was missing – what about broken English?
There are times when people switch between languages and use a mix of words that would not be understood by a normal chatbot. Our goal isn't just to find something that speaks one of South Africa's 11 official languages, but to find out what someone really is trying to convey.” says Brent.
“I was astonished that there was no huge waiting list to get into this platform and I found the company's vision quite inspiring. I signed up and started playing around on the playground.”
It’s important to be able to interact with the models, explore presets, and play around before committing to an NLP.
“I am not a coder,” he adds. “When I saw that AI21 Studio had an easy-to-use fine-tuning feature that did not require any previous coding experience, I was excited to see how quickly I could prototype different conversational flows and build something that felt human and relatable.”
Beyond the Chatbot: How Harambee Used AI21 to Reach Their End Goal
Harambee saw incredible success using the chatbot.
“The conversations [work seekers are having] are incredible,” says Brent. “For the first time, they feel like they are important. For the first time, they feel like their goals are important. They feel like there is someone there to support them.”
Beyond building rapport and genuine connection with the South African youth, they were able to utilize AI21 to make their jobs easier and improve further processes.
Solution #1: Content creation through “lifehacks”
With AI copywriting, you can automate repetitive writing tasks. Brent and the Harambee team were able to develop around 60 different "lifehacks" for users who had signed up for Coachmee check-in sessions.
Lifehacks have to strike a careful balance between being aspirational and transcendent, while being practical and relevant. All within a few sentences, making sure someone doesn't have to scroll back up to read the full message.
Another unique contribution of AI21 Labs was around the use of emojis, which enabled the lifehacks to be much more playful and engaging.
Solution #2: Unique reference letters
With the summarization tool, you can compress documents and extract key insights into short summaries. Brent and the Harambee team used this to create 100% unique, relevant reference letters for each individual who finished their training.
“Sometimes the model can go way too far and make up crazy things about the person,” says Brent, discussing what has gone wrong while using other applications.
“[But] when a user completes 20 check-ins with Coachmee, we have enough data that can factually testify to someone's commitment to growth and goal-achievement. That information, together with the deep insights into their aspirations and personality that we lifted from the initial sign-up conversation, we are able to use Studio API to generate beautiful reference letters that they can attach to their CVs and take with them to interviews,” explains Brent. “I [once] got a message saying, ‘I gotta frame this!’”
The Result: Helping Thousand of Youths Gain Employment When They Need it Most
Harambee has fine-tuned its AI21 NLP experience to drive meaningful conversations. Their data shows that the stronger the rapport that is built at the beginning of the relationship, the more chance the person has at continuing and completing the process.
Brent concludes that “compared to the first iteration of Coachmee which did not use an NLP for the sign-up process, our conversion rate from sign-up to completion has improved by 20%.”
In addition, he tells us that almost 1,000 young people have measurably higher prospects of employment thanks to the unique reference letters that AI21 Studio API helped create. “AI21 has been the most generous, collaborative and supportive technology partner that I have ever worked with.”
We are thrilled to have the opportunity to use our NLP technology to help elevate the careers of thousands of youths in Africa. We hope this project serves as inspiration to many other organizations around the world.
ABOUT THE AUTHOR
Stay up to date with the latest research and updates from AI21 Labs.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
In August 2021 we released Jurassic-1, a 178B-parameter autoregressive language model. We’re thankful for the reception it got – over 10,000 developers signed up, and hundreds of commercial applications are in various stages of development. Mega models such as Jurassic-1, GPT-3 and others are indeed amazing, and open up exciting opportunities. But these models are also inherently limited. They can’t access your company database, don’t have access to current information (for example, latest COVID numbers or dollar-euro exchange rate), can’t reason (for example, their arithmetic capabilities don’t come close to that of an HP calculator from the 1970s), and are prohibitively expensive to update. A MRKL system such as Jurassic-X enjoys all the advantages of mega language models, with none of these disadvantages. Here’s how it works.
Compositive multi-expert problem: the list of “Green energy companies” is routed to Wiki API, “last month” dates are extracted from the calendar and “share prices” from the database. The “largest increase“ is computed by the calculator and finally, the answer is formatted by the language model.
There are of course many details and challenges in making all this work - training the discrete experts, smoothing the interface between them and the neural network, routing among the different modules, and more. To get a deeper sense for MRKL systems, how they fit in the technology landscape, and some of the technical challenges in implementing them, see our MRKL paper. For a deeper technical look at how to handle one of the implementation challenges, namely avoiding model explosion, see our paper on leveraging frozen mega LMs.
A further look at the advantages of Jurassic-X
Even without diving into technical details, it’s easy to get a sense for the advantages of Jurassic-X. Here are some of the capabilities it offers, and how these can be used for practical applications.
Reading and updating your database in free language
Language models are closed boxes which you can use, but not change. However, in many practical cases you would want to use the power of a language model to analyze information you possess - the supplies in your store, your company’s payroll, the grades in your school and more. Jurassic-X can connect to your databases so that you can ‘talk’ to your data to explore what you need- “Find the cheapest Shampoo that has a rosy smell”, “Which computing stock increased the most in the last week?” and more. Furthermore, our system also enables joining several databases, and has the ability to update your database using free language (see figure below).
Jurassic-X enables you to plug in YOUR company's database (inventories, salary sheets, etc.) and extract information using free language
AI-assisted text generation on current affairs
Language models can generate text, yet can not be used to create text on current affairs, because their vast knowledge (historic dates, world leaders and more) represents the world as it was when they were trained. This is clearly (and somewhat embarrassingly) demonstrated when three of the world’s leading language models (including our own Jurassic-1) still claim Donald Trump is the US president more than a year after Joe Biden was sworn into office. Jurassic-X solves this problem by simply plugging into resources such as Wikidata, providing it with continuous access to up-to-date knowledge. This opens up a new avenue for AI-assisted text generation on current affairs.
Who is the president of the United States?
Joe Biden is the 46th and current president
Jurassic-X can assist in text generation on up-to-date events by combining a powerful language model with access to Wikidata
Performing math operations
A 6 year old child learns math from rules, not only by memorizing examples. In contrast, language models are designed to learn from examples, and consequently are able to solve very basic math like 1-, 2-, and possibly 3- digit addition, but struggle with anything more complex. With increased training time, better data and larger models, the performance will improve, but will not reach the robustness of an HP calculator from the 1970s. Jurassic-X takes a different approach and calls upon a calculator whenever a math problem is identified by the router. The problem can be phrased in natural language and is converted by the language model to the format required by the calculator (numbers and math operations). The computation is performed and the answer is converted back into free language. Importantly (see example below) the process is made transparent to the user by revealing the computation performed, thus increasing the trust in the system. In contrast, language models provide answers which might seem reasonable, but are wrong, making them impractical to use.
The company had 655400 shares which they divided equally among 94 employees. How many did each employee get?
Each employee got 7000 stocks
(No answer provided)
6972.3 X= 655400/94
Jurassic-X can answer non-trivial math operations which are phrased in natural language, made possible by the combination of a language model and a calculator
Solving simple questions might require multiple steps, for example - “Do more people live in Tel Aviv or in Berlin?” requires answering: i. What is the population of Tel-Aviv? ii. What is the population of Berlin? iii. Which is larger? This is a highly non-trivial process for a language model, and language models fail to answer this question (see example). Moreover, the user can’t know the process leading to the answers, hence is unable to trust them. Jurassic-X can decompose such problems into the basic questions, route each to the relevant expert, and put together an answer in free language. Importantly, Jurassic-X not only provides the correct answer but also displays the steps taken to reach it, increasing the trust in the system.
Do more people live in Tel Aviv or in Berlin?
There are more people living in Tel Aviv than in Berlin.
Berlin and Tel Aviv are roughly the same size
(First hit is a comparison between Tel Aviv and Berlin)
More people live in Berlin than in Tel-Aviv
[‘Return population of Tel Aviv’; Return population of Berlin’; Return which is bigger between #1 and #2’] Step 1: Population of Tel Aviv. Result - 451523. Step 1: Population of Berlin. Result - 3664088. Step 3: Which is bigger, #1 or #2. Result - Berlin.
Jurassic-X breaks down compositional questions, answers the basic sub-questions, and puts together the answer. Importantly, this process is transparent to the user greatly increasing the trust in the system
Dynamic information (like weather and currency exchange rates)
Certain types of information change continuously - weather, currency exchange rates, share values and more. Such information will never be captured by language models, yet can easily be handled by Jurassic-X by integrating it with a reliable source of information. We performed a proof-of-concept on two such features - weather and currency exchange rates, and the design enables quick integration with more sources to solve your use-case. Weather - a loosely phrased question about the weather elicits an answer from all language models, where language models always return the same answer, regardless of when the question was asked (funny, right?), while Jurassic-X provides an answer based on the actual weather prediction.
I’m going to be in New-York in 3 days. Should I pack my umbrella?
Yes, you should pack your umbrella.
Yes, you should. The weather forecast is rain.
(Links to weather websites)
Yes, you should pack your umbrella, because in New York in 3 days there will be broken clouds and the temperature will be -2 degrees.
Currency exchange rates change much faster than weather predictions, yet the Jurassic-X concept - a language model connected to a reliable source of information - easily solves this problem as well.
How much Moroccan money will I get for 100 bucks?
100 moroccan dirhams is about 27$.
How much is 100 dollars in moroccan money?
100 dirhams = 10.75 dollars
100 USD = 934.003 MAD
Jurassic-X combines a language model with access to APIs with continuously changing information. This is demonstrated for weather forecasts and currency exchange rates, and can easily be extended to other information sources
Transparency and trust
Transparency is a critical element that is lacking in language models, preventing a much wider adoption of these models. This lack of transparency is demonstrated by the answers to the question - “Was Clinton ever elected as president of the United States?”. The answer, of course, depends on which Clinton you have in mind, which is only made clear by Jurassic-X that has a component for disambiguation. More examples of Jurassic-X’s transparency were demonstrated above - displaying the math operation performed to the user, and the answer to the simple sub-questions in the multi-step setting.
Was Clinton ever elected president of the United States?
No, Clinton was never elected as president of the United States.
Clinton was elected president in the 1992 presidential elections…
Bill Clinton was elected president.
Jurassic-X is designed to be more transparent by displaying which expert answered which part of the question, and by presenting the intermediate steps taken and not just the black-box response
That's it, you get the picture. The use cases above give you a sense for some things you could do with Jurassic-X, but now it's your turn. A MRKL system such as Jurassic-X is as flexible as your imagination. What do you want to accomplish? Contact us for early access
Your submission has been received!
Oops! Something went wrong while submitting the form.