Entity Recognition with Azure Cognitive Services in C#
Building intelligent apps by adding context to text
Entity recognition is a powerful way to analyze blocks of text and highlight key people, locations, or topics of interest from that sentence for an artificial intelligence application to respond to. Azure Cognitive Services gives us standard entity recognition and linked entity recognition where we can get links to well-known sources of information about the entities mentioned in the text. In this article we’ll cover both technologies and how to work with them with C# or generic REST messages.
In order to get the most out of this article you should be familiar with:
- The basics of Azure Cognitive Services
- The basics of C# programming
- How to add a package reference using NuGet
Adding a Reference to the Azure Cognitive Services Language SDK
In order to recognize entities, we should add a reference to the Azure SDK. To get this we’ll install the latest version of the Azure.AI.TextAnalytics package in Visual Studio using NuGet package manager.
Caution: do not use
Microsoft.Azure.CognitiveServices.Language.TextAnalytics. This is the old version of this library and it has some known bugs.
See Microsoft’s documentation on NuGet package manager for additional instructions on adding a package reference.
Note: Adding the package can also be done in the .NET CLI with
dotnet add package Azure.AI.TextAnalytics.
Creating a TextAnalyticsClient Instance
We need some using statements at the beginning of our C# file to work with the TextAnalytics namespace:
using Azure;
using Azure.AI.TextAnalytics;
After that, we’ll store key and endpoint. These can be found on the Keys and Endpoints blade of your cognitive services instance in the Azure portal.
If you are using a single Azure Cognitive Services instance, you should use one of that service’s keys and its endpoint. If you wanted an isolated service and created a stand-alone Language Service, you would use that service’s key and endpoint instead.
// These values should come from a config file and should NOT be stored in source control
string key = "YourKeyGoesHere";
Uri endpoint = new Uri("https://YourCogServicesUrl.cognitiveservices.azure.com/");
Important Security Note: In a real application you should not hard-code your cognitive services key in your source code or check it into source control. Instead, you should get this key from a non-versioned configuration file via
IConfigurationor similar mechanisms. Checking in keys can lead to people discovering sensitive credentials via your current source or your source control history and potentially using them to perform their own analysis at your expense.
Next, we’ll set up a TextAnalyticsClient. This object will handle all communications with Azure Cognitive Services later on.
// Create the TextAnalyticsClient and set its endpoint
AzureKeyCredential credentials = new AzureKeyCredential(key);
TextAnalyticsClient textClient = new TextAnalyticsClient(endpoint, credentials);
With the textClient created and configured, we’re ready to start recognizing entities.
Entity Recognition
Next we need some text to perform entity recognition on. I’m a fan of musicals, so we’ll use the opening verse of the Major-General’s song from Gilbert and Sullivan’s The Pirates of Penzance.
const string text = "I am the very model of a modern Major-General. " +
"I've information vegetable, animal, and mineral. " +
"I know the kings of England, and I quote the fights Historical, " +
"from Marathon to Waterloo, in order categorical.";
Next, we’ll run the following C# code to identify entities:
// Detect Entities
Response<CategorizedEntityCollection> recognizeResponse = textClient.RecognizeEntities(text);
This will send a REST POST request to https://YourCogServicesUrl.cognitiveservices.azure.com/text/analytics/v3.1/entities/recognition/general?stringIndexType=Utf16CodeUnit with the following body:
{
"documents":[
{
"id":"0",
"text":"I am the very model of a modern Major-General. I\u0027ve information vegetable, animal, and mineral. I know the kings of England, and I quote the fights",
"language":"en"
}
]
}
Note that the language defaulted to “en” for English. If you wanted to, you could have specified a different language code as a parameter to RecognizeEntities. Of course, you could also use the language detection features of cognitive services to provide this based on a prior call to the cognitive services API.
Microsoft also gives us an RecognizeEntitiesBatch method that allows you to specify multiple strings instead of one string. Additionally, there are Async versions of both RecognizeEntities and RecognizeEntitiesBatch.
Another important note is that this REST request includes an Ocp-Apim-Subscription-Key header containing your cognitive services key. This is how Microsoft knows that you’re allowed to make the request to that cognitive services endpoint. If you fail to include this header or use the wrong key, your call will not succeed.
Understanding Entity Results
Let’s take a look at the JSON response we got to the REST request we made earlier:
{
"documents":[
{
"id":"0",
"entities":[
{
"text":"Major-General",
"category":"PersonType",
"offset":32,
"length":13,
"confidenceScore":0.97
},
{
"text":"vegetable",
"category":"Product",
"offset":64,
"length":9,
"confidenceScore":0.9
},
{
"text":"mineral",
"category":"Product",
"offset":87,
"length":7,
"confidenceScore":0.9
},
{
"text":"kings",
"category":"PersonType",
"offset":107,
"length":5,
"confidenceScore":0.95
},
{
"text":"England",
"category":"Location",
"subcategory":"GPE",
"offset":116,
"length":7,
"confidenceScore":0.99
}
],
"warnings":[]
}
],
"errors":[],
"modelVersion":"2021-06-01"
}
That’s a lot of information, and we note that some categories such as Locations also contain sub-categories. Note that each entity has the text of the entity from the original sentence, a category, a confidence score, and information on where in the text it appeared.
We can log the information using the Azure SDK as follows:
CategorizedEntityCollection entities = recognizeResponse.Value;
Console.WriteLine();
Console.WriteLine("Entities:");
foreach (CategorizedEntity entity in entities)
{
Console.WriteLine($"\t{entity.Text} (Category: {entity.Category}) with {entity.ConfidenceScore:P} confidence");
}
This would display the following text:
Entities:
Major-General (Category: PersonType) with 97.00% confidence
vegetable (Category: Product) with 90.00% confidence
mineral (Category: Product) with 90.00% confidence
kings (Category: PersonType) with 95.00% confidence
England (Category: Location) with 99.00% confidence
So here we see the various entities encountered in this passage of text and even the category they belong to. This can help us programmatically make decisions about what the user may be intending (however Conversational Language Understanding is more suited to this task).
Linked Entities
But wait; there’s more! Azure Cognitive Services also gives us the ability to get linked entities back, which contain links directly to a knowledgebase article of interest on the topic. This allows us to either make an automated request for more information or direct the user towards additional sources of information based on the text that was analyzed.
The initial code for detecting linked entities is fairly similar to the entity recognition call we saw earlier:
// Detect Linked Entities
Response<LinkedEntityCollection> linkedResponse = textClient.RecognizeLinkedEntities(text);
And, indeed, the POST it does to https://YourCogServicesUrl.cognitiveservices.azure.com/text/analytics/v3.1/entities/linking?stringIndexType=Utf16CodeUnit is very similar to the one we saw earlier as well with an identical message body:
{
"documents":[
{
"id":"0",
"text":"I am the very model of a modern Major-General. I\u0027ve information vegetable, animal, and mineral. I know the kings of England, and I quote the fights Historical, from Marathon to Waterloo, in order categorical.",
"language":"en"
}
]
}
However, the response it receives is different:
{
"documents":[
{
"id":"0",
"entities":[
{
"bingId":"defb21d9-088d-154e-aef6-36d932b8741b",
"name":"Major-General's Song",
"matches":[
{
"text":"I am the very model of a modern Major-General",
"offset":0,
"length":45,
"confidenceScore":0.72
}
],
"language":"en",
"id":"Major-General's Song",
"url":"https://en.wikipedia.org/wiki/Major-General's_Song",
"dataSource":"Wikipedia"
},
// (others omitted for brevity)
{
"bingId":"9cb0bc53-d917-98be-3703-db9c51d7698b",
"name":"The Fifteen Decisive Battles of the World",
"matches":[
{
"text":"the fights Historical",
"offset":137,
"length":21,
"confidenceScore":0.8
}
],
"language":"en",
"id":"The Fifteen Decisive Battles of the World",
"url":"https://en.wikipedia.org/wiki/The_Fifteen_Decisive_Battles_of_the_World",
"dataSource":"Wikipedia"
}
],
"warnings":[]
}
],
"errors":[],
"modelVersion":"2021-06-01"
}
With this we can see the wealth of additional information coming back on the matches, including the name, ID, and link to the relevant content.
While preparing this article I was very surprised to find the link to the fifteen decisive battles of the world generated from Gilbert and Sullivan. However, the link makes sense in context and is a small-scale example of being pleasantly surprised by artificial intelligence.
We can loop over our linked entities and display them, just like we did with normal entities:
Console.WriteLine("Linked Entities:");
// Loop over the linked entities recognized
LinkedEntityCollection linkedEntities = linkedResponse.Value;
foreach (LinkedEntity entity in linkedEntities)
{
Console.WriteLine($"\t{entity.Name} (Url: {entity.Url})");
// Each linked entity may have multiple matches
foreach (LinkedEntityMatch match in entity.Matches)
{
Console.WriteLine($"\t\tMatched on '{match.Text}' with {match.ConfidenceScore:P} confidence");
}
}
However, with linked entities, each entity may have multiple matches associated with it from the source sentence. For example, a sentence may refer to Seattle twice in the same sentence. If that were the case it would likely show up twice in Matches.
Here’s what our code above displays:
Linked Entities:
Major-General's Song (Url: https://en.wikipedia.org/wiki/Major-General's_Song)
Matched on 'I am the very model of a modern Major-General' with 72.00% confidence
Animal (Url: https://en.wikipedia.org/wiki/Animal)
Matched on 'animal' with 10.00% confidence
England (Url: https://en.wikipedia.org/wiki/England)
Matched on 'England' with 0.00% confidence
The Fifteen Decisive Battles of the World (Url: https://en.wikipedia.org/wiki/The_Fifteen_Decisive_Battles_of_the_World)
Matched on 'the fights Historical' with 80.00% confidence
Marathon (Url: https://en.wikipedia.org/wiki/Marathon)
Matched on 'Marathon' with 2.00% confidence
Waterloo, Iowa (Url: https://en.wikipedia.org/wiki/Waterloo,_Iowa)
Matched on 'Waterloo' with 7.00% confidence
Now, I am chuckling that Azure Cognitive Services thinks the major-general was singing about Waterloo, Iowa, but to be fair to entity recognition, I did tell it that the language we were speaking was English and it is only 7% confident in this linkage.
All told, entity recognition is a powerful capability to analyze text and provide additional points of interest, possibly including links to places where we can learn more about each topic of interest. I could easily see this technology being used to make an educational app or enrich a chatbot by letting it provide additional links to learn more about something.
