Understand SharePoint Crawled and Managed properties for search

Understand SharePoint Crawled and Managed properties for search

by

When you start learning SharePoint, they teach you about columns and site columns. In SharePoint 2013, the one thing everyone talks about is the new search and what can be done with it. However, rarely do we start from the beginning and explain the basics. To start anything with search related features, you will need to understand crawled and managed properties and the difference between the two.

The basics of SharePoint search

One of the biggest issues I see with SharePoint, is how little organizations know about search and the time and effort that needs to go into it. There is a lot that goes into the configuration alone and I highly recommend you investigate things like the topology, content sources and crawl schedules amongst many other things. But for the purpose of this article, let’s cover the basics of crawling with search for SharePoint.

Pretend for a second that the building where you work is your SharePoint farm. It’s brand new and completely empty, the rooms represent your SharePoint sites. To be able to ask the search engine what you have in each of these rooms, you will first need it to run around the building and take notes of everything. This is what we call crawling. How often the crawling happens will determine your search result “freshness” amongst other settings like the crawl type.

Understand SharePoint Crawled and Managed properties for search

At this point, many people assume that the columns they created in their lists and libraries are now searchable. However, in most cases it is still not the case.

SharePoint Search Crawled Properties

Though there are always exceptions and I will get to them shortly, once the indexer passes and crawls your content and its metadata it adds the columns as crawled properties only. This means it has passed over your columns and the metadata assigned to each element. You do not have any control over the creation of “Crawled Properties”. These are the resulting list of columns crawled in some way, though I am over simplifying it.

A crawled property by itself is useless for you when trying to run or build search queries or even display the value of this property in search results. Picture the crawled properties more as the information collected by that crawl that has been going around your building in our previous example. The crawl goes through your sites, lists and libraries to find your content and picks up the value in your columns and stores them as crawled properties.

SharePoint Search Managed Properties

The Managed Properties is where it all happens. If you need anything to be search related: Search Results, Content Search, Refiners, Display Templates, etc. Then you will need to understand how to create and manage these “Managed Properties”.

Though they have many features and configurations to them, at the core, they are the grouping of one or multiple “Crawled Properties”. Let me explain, pretend that your organization has many lists and libraries. Ok, maybe you don’t have to pretend. And in them, users created columns like “Customer Name” and “Client”. For the organization, these two columns represent exactly the same content, but not for search. For search, they are just crawled properties and two very different ones at that because they do not share the same name. On top of that, since they are only Crawled Properties, if someone searches for all documents where Client=Sharegate then they will find nothing at all. Because no search related feature works with crawled properties themselves, for that we will need to create a Managed Property called “Customer” and assign both of those crawled properties to it.

In some scenarios though, you may find that a Managed Property has already been created for your Crawled Property, automatically. Well that’s because as always, there are exceptions. If you create a Site Column and assign it to a list or library, when the indexer crawls over it, it will automatically create a Managed Property for it. Also, regardless of it being a site column or not, Managed Metadata columns will always have a Managed Property created for them.

Where do I create and manage these properties?

Now that we understand the difference between the two at a high level, we’ll need to know how to see the crawled properties collected and created by our search engine as well as the existing managed properties. Luckily for us, in SharePoint 2013 we no longer have to go in the Central Administration of SharePoint or Office 365 to create them. Of course, it is still what I would recommend as these will be global and centralized for all web applications associated to this search service application. However, it’s not unlikely for certain Site Collections to need their own Managed Properties and isolate them there.

To navigate, edit or create these properties we simply need to find the Search Schema either in the Central Administration for the search service application or under site settings of a site or site collection.

Understand SharePoint Crawled and Managed properties for search

The problem with optimistictitleoverride and the title managed property

I will take no credit for this one as I learned this from one of the kings of SharePoint Search, Mikael Svenson. Check out his article on "What makes a SharePoint column searchable". The problem we’ve been experiencing with SharePoint Search is it’s over willingness to help improve search results. You see, there is an integrated feature built in search that will extract what it thinks is the Title of your document and show that in the Search results, regardless of the value you entered in the Title column. The search engine will take for example your headings in a Word document or the first slide with large text in PowerPoint and override any other title you may have specified. This was very unpleasant as you can imagine, many documents would often have the same heading as they were the result of a filled out form. As a result, SharePoint would hide our documents in search results calling them duplicates.

In SharePoint 2013, with the October 2013 CU or SP1 installed on your server, you will now be able to see this special “Crawled Property” in your Title Managed Property.

Understand SharePoint Crawled and Managed properties for search

Remember, SharePoint Search Results and any other search related feature can only show and work with managed properties. So if the Title being displayed is the wrong one, then we need to go see the Managed Property responsible for it and see what crawled properties are assigned to it and in what order of priority. In the past, we could not see this special crawled property that had the highest priority and in turn could not adjust it. Thankfully, that is no longer and issue. Thank you Mikael!

Before upgrading SharePoint with a new Search solution

SharePoint crawled and managed properties are necessary to understand for you to have a successful and meaningful search engine. Many times we like to kick and scream at SharePoint for not providing us with meaningful results, but in most cases it is due to a lack of configuration. One of the reasons we choose to migrate to SharePoint 2013 is to take advantage of these new search features and the benefits they bring us. To build search-driven sites using the new Content Search Web Part or the Catalog feature to integrate them straight to a publishing site. But if we want to take complete control of these features and what they are showing our visitors, we first need to understand these basics. There is another great explanation on TechNet on crawled and managed properties you can read. Once you feel you have grasped this concept, you will be ready to attack some cool concepts and build awesome things with SharePoint. I encourage you to visit these 30 helpful SharePoint articles I grouped together, they include recordings and free display templates you can download and play with.

Benjamin Niaulin
Benjamin Niaulin @bniaulin

Well known as the SharePoint Geek, Benjamin has been helping people all around the globe reach their goals by simplifying SharePoint solutions. You haven't met Benjamin yet? Look for him at SharePoint conferences and events!