Add a custom field in the External Index

Questions
1

Hi,

I want to extend my externalindex with some fields. For instance if we are looking at a TAG datatype field, the tags are comma separated… This is not working for facetting so i want to add a field tag_face, where i can index each value separately. So the index will contain each tag non tokenized, as a tag could be multiple words.

I see that we can add some field definitions :

`options.FieldDefinitions.AddOrUpdate(new FieldDefinition(“price”, FieldDefinitionTypes.Double));`

But there is no FieldDefinitionsTypes for Array for instance. And even i can not map a field to a new name. For instance i maybe still need the original “tags” definitions, but i want to add “tags_facets” for the tags field of the content type.

Thanks, and enjoy.

4

hi

Which Umbraco version are you on?

5

You can add your own custom fields to the index :

Custom indexing | CMS | Umbraco Documentation

You can then take the tags values and save them in any format you like, here is a sample of how I’ve handled tags in the past:

				case FaqItem.ModelTypeAlias:
					//Setting searchableTags on FAQ items
					if (contentNode.HasValue(AliasConstants.FAQItem.Tags))
					{
						var hasTags = e.ValueSet.Values.TryGetValue(ExamineConstants.FieldNames.Tags, out var existingTags);
						if (!hasTags)
						{
							return;
						}
						var tagsLowercase = existingTags!.Select(t => t.ToString()?.ToLowerInvariant());
						var newValues = new List<object>();

						foreach (var tag in tagsLowercase)
						{
							if (!string.IsNullOrWhiteSpace(tag))
							{
								newValues.Add(tag.AddLuceneDelimiterWord());
							}
						}

						updateValues.Add(ExamineConstants.FieldNames.SearchableTags, newValues);
					}
					break;
}

This is in Umbraco 17 but previous versions should handle it in a similar way.
This saves the tags in to a new field that I’ve named SearchableTags.

6

Thanks Owain.

Yes I can add FieldDefinitions from the documentation I read. But where do I plug custom indexing ? This is not in the documentation. And it’s a bit overload to create a new index just to add one or two fields that can be used for facetting efficiently with Lucene.

7

Hi,
You don’t need a new index, you can add the field to ExternalIndex.

In the docs there is

 if (name.Equals(Constants.UmbracoIndexes.ExternalIndexName))
        {
            options.FieldDefinitions.AddOrUpdate(new FieldDefinition("price", FieldDefinitionTypes.Double));
        }

This adds the new custom field to the ExternalIndex.

You would need to add a your custom code to a composer but that is also covered in the docs.

Does that make sense?

8

Yes, I read that one. But there is no FieldDefinitions types for multiple values. I do not want multiple values stored as CSV, i need to index each value of a field. This is how we can work efficiently with Lucene and facetting.
So how can i change the indexing, by providing my own logic and my own fields… as FieldDefinitionTypes is lacking of some features. I even don’t know why it has not been implemented, the core fields like __Path should already be implemented with an additional field where we can just find all docs where the path contains a certain nodeId, and not going through like search all nodes with *,NodeId,* or endsWith *,NodeId. Same for tags or any multiple values

9

Hi @cmwalolo
Here is the code I use to search for tags on a project I have just now.
Once you have the tags in their own field in Examine (External Index) when you search from the frontend you can call something like this :

public IEnumerable<FaqItem> GetFAQsByTags(IList<string>? tags, IPublishedContent contextPage)
{
	if (tags == null || tags.Count == 0)
	{
		return Enumerable.Empty<FaqItem>();
	}

	var searchPathIds = GetSearchPathIds(contextPage).ToList();
	var allFaqItems = new List<FaqItem>();

	foreach (var tag in tags)
	{
		var faqItems = SearchFAQsByTag(contextPage, tag, searchPathIds);
		allFaqItems.AddRange(faqItems);
	}

	return allFaqItems.GroupBy(faq => faq.Id).Select(group => group.First());
}



private IEnumerable<FaqItem> SearchFAQsByTag(IPublishedContent contextPage, string tag, IEnumerable<string> searchPathIds)
{
	if (!_examineManager.TryGetIndex(UmbracoIndexes.ExternalIndexName, out var index))
	{
		_logger.LogWarning("External index not found when searching for FAQ tag: {Tag}", tag);
		return Enumerable.Empty<FaqItem>();
	}

	try
	{
		var searcher = index.Searcher;
		var query = searcher.CreateQuery("content");
		var tagQuery = query.Field(ExamineConstants.FieldNames.SearchableTags, tag.AddLuceneDelimiterWord().ToLowerInvariant().Escape());

		if (searchPathIds.Any())
		{
			tagQuery = tagQuery.And().GroupedOr(["searchPath"], searchPathIds.ToArray());
		}

		var results = tagQuery.Execute(QueryOptions.SkipTake(0, 9999));
		return results.Select(x => _umbracoHelper.Content(x.Id)).OfType<FaqItem>();
	}
	catch (Exception ex)
	{
		_logger.LogError(ex, "Error searching for FAQ items with tag: {Tag}", tag);
		return Enumerable.Empty<FaqItem>();
	}
}