BERT stands for Bidirectional Encoder Representation form Transformers and was developed in 2018 by Google.  This NLP technique utilized pre-trained  transformer type neural networks. There are different sizes of the BERT model to meet  various use-cases and available computing resources for different applications.  The model size does influence the the accuracy of the predictions.

In general having a larger model size results in a more accurate result. However, larger models consume more processing power and take longer to process. In the case of conversation AI processing delays can greatly impact the user experience and studies have shown that in a typical natural language conversation with the average delay in between exchanges is 300ms. This is a very narrow window of time to evaluate the intent of the user, fetch any external data that is required and prepare a response. When you are running multiple models against a query you may only have 10ms to evaluate and decode the natural language. Source:https://blogs.nvidia.com/blog/2021/02/25/what-is-conversational-ai/

According to Mohd Sanad Zaki RizviSource: https://wandb.ai/jack-morris/david-vs-goliath/reports/Does-Model-Size-Matter-A-Comparison-of-BERT-and-DistilBERT--VmlldzoxMDUxNzU

TinyBERT model to achieve 96% of its BERT base teacher on the BLUE benchmark while being 7.5x smaller and 9.4x faster! Its performance numbers are impressive even when comparing with BERT small, a model of exactly the same size, which TinyBERT is 9% better than (76.5 vs 70.2 points average on GLUE).

Table of published pre-trained BERT models

GPT stands for Generative Pre-trained Transformer and as of January 2022  is in it’s third generation. GPT is an auto-regressive language model that uses deep learning to produce human-like text.  GPT-3 utilizes a total of 175 billion of parameters it is one of the largest publicly available language models available. Although the model is now owned by Microsoft there are a number of API interfaces to the model for a varieties of uses.

In general GTP-3 is advertised to be able to be “applied to virtually any task that involves understanding or generating natural language or code. We offer a spectrum of models with different levels of power suitable for different tasks, as well as the ability to fine-tune your own custom models. These models can be used for everything from content generation to semantic search and classification.”Source: https://beta.openai.com/docs/introduction/overview

GPT-3 Key Features

In general there are 4 key features of GPT-3. These are

• Completion
• Classification
• Search

According to the GPT-3 DocumentationSource: https://beta.openai.com/docs/guides/completion/prompt-design there are number of API endpoints that can be accessed within each of these three key features.

Completion

Completion provides a simple but powerful interface whereby the user provides some textual input as a prompt, and the model will generate a text completion that attempts to match whatever context or pattern you gave it.

Generation – One of the most powerful yet simplest tasks you can accomplish with the API is generating new ideas or versions of input. You can ask for anything from story ideas, to business plans, to character descriptions and marketing slogans. In this example, we’ll use the API to create ideas for using virtual reality in fitness.

Conversation – The API is extremely adept at carrying on conversations with humans and even with itself. With just a few lines of instruction, we’ve seen the API perform as a customer service chatbot that intelligently answers questions without ever getting flustered or a wise-cracking conversation partner that makes jokes and puns. The key is to tell the API how it should behave and then provide a few examples.

Transformation – The API is a language model that is familiar with a variety of ways that words and characters can be used to express information. This ranges from natural language text to code and languages other than English. The API is also able to understand content on a level that allows it to summarize, convert and express it in different ways.

Summarization – The API is able to grasp the context of text and rephrase it in different ways. In this example, we create an explanation a child would understand from a longer, more sophisticated text passage. This illustrates that the API has a deep grasp of language.

Factual responses – The API has a lot of knowledge that it’s learned from the data that it was been trained on. It also has the ability to provide responses that sound very real but are in fact made up. There are two ways to limit the likelihood of the API making up an answer.

Classification 

Can be used to evaluate future data against a number of criteria such as  positive and negative. The Classifications endpoints provides the ability to leverage a labeled set of examples without fine-tuning and can be used for any text-to-label task. By avoiding fine-tuning, it eliminates the need for hyper-parameter tuning. The endpoint serves as an “autoML” solution that is easy to configure, and adapt to changing label schema. Up to 200 labeled examples or a pre-uploaded file can be provided at query time.

Search

The Search endpoint  allows you to do a semantic search over a set of documents. This means that you can provide a query, such as a natural language question or a statement, and the provided documents will be scored and ranked based on how semantically related they are to the input query.

Natural Language Platforms

Using an established Natural Language Platforms make it easier for developers to design and deploy conversational systems to users as well as helps developers reach more users and also the capability to interface to Internet of Things (IoT) devices.

Most natural language processing platforms today handle the two key aspects related to natural language processing and understanding. They more or less all follow a similar methodology whereby they interpret a users utterance, then map it against intents that are created in order to respond to the user’s utterance.

These Conversational Language Platform benefit from having a huge amount of computing resources  powering the back-end which is therefore capable of supporting a larger language model, as well as benefit from having a diverse language set from  a wide range of users from around the world. As a result these web service frameworks tend to offer improved natural language understanding capabilities when compared with a stand-alone system.

Amazon Lex

Google Dialogflow

Microsoft LUIS

To access Dialogflow you first need to have a Google account. Then you can navigate to the Dialogflow console which can be found here.

https://dialogflow.cloud.google.com/

Once you are logged into the Dialogflow console (figure 1) you will be prompted to create a new agent.  (figure 2)

This agent represents your application and can be thought of as the personification of a human agent that will converse with your end users.  You will be required to provide the following attributes as you create the project.

• Agent Name
• Default Language
• Default Timezone.

The default language in time zone are used unless overridden by the client.

Note: You cannot change the default language after you create the agent.

As of Janauary 2022, Dialogflow supports 55 language variations for text chats. It is important to note that not all languages are fully supported for all client types. Since Dialogflow is capable of a number of different client features that are listed below it is important to consult the Official Dialogflow Language table to make sure that your languages is supported for the client features that you are looking to use.

Dialogflow Client Features

• Text (text-only chat)
•   STT (speech-to-text, audio input, speech recognition)
•   TTS (text-to-speech, audio output, speech synthesis)
• Phone (Dialogflow Phone Gateway)
• Knowledge (Knowledge Connectors)
• Sentiment (Sentiment Analysis)
• SmTalk (Built-in Small Talk)

Dialogflow Menu

The menu on the left side of the screen provides quick access to the majority of the Dialogflow features.

Intents – This is where you will manage the intents of your agent. And intent a way of handling the end-users intention during a conversation. It could be thought of as a reason that they are conversing with your agent or as something they would like to do during the conversation.

Entities – this is where you will manage the entities of your agent. Entities can be thought of as data variables that the user provides during conversation.

Knowledge – the knowledge feature is used to connect to an external data source such as a webpage and can automatically create things like a set of frequently asked questions or knowledge base.

Fulfillment – This is how Dialogflow connects to different back-end systems to extract information. Such as retrieving current weather conditions from a weather service API.

Integrations- This is where your Digalogflow agent connects with other platforms such as Slack, Facebook Messenger, Twitter, Telegram or even the telephone system.

The rest of this menu will be discussed in future chapters of this book.