Make the most of SharePoint search APIs

Our organizations thrive on content, with documents still at the heart of our myriad workflows. Once those documents have served their initial purpose, they’re still important; those forms, reports, and papers all capture essential business knowledge. Although we could just dump them in the digital equivalent of trash, it’s far better to take advantage of low-cost storage, either on premises or in the cloud, to build digital hoards of knowledge that might come in useful someday.

Those digital troves are nothing like old paper archives. The more information they store, the more useful they become, providing a pool of knowledge that lets software tease out links in our corporate knowledge graph and provide insights into what we know and who knows it. But we can’t do that without smart stores and effective search tools.

Enterprise content management tools, such as Microsoft’s SharePoint, are nothing like the content management platforms powering sites like InfoWorld. Although they do create a framework for displaying and managing content, they’re better thought of as a digital librarian, providing tools for cataloging and searching files. You can think of them as specialized databases, a precursor of today’s NoSQL document databases, working with large text files of unstructured content. Instead of only working with JSON, though, SharePoint is optimized to work with Office content.

Back when Microsoft was developing what became Office 365, it was clear that SharePoint was going to be its Office Server, a counterpart to Windows Server’s storage tools. Today it’s one of the hubs of Microsoft 365, underpinning Lists, parts of Teams, and the enterprise OneDrive. It’s also a development platform in its own right, allowing you to build and deploy intranet applications using its own development tools and the Microsoft Graph.

Search in SharePoint

Perhaps the most important tool for building any SharePoint application after workflow is search. This is how your users will find documents and content and get relevant results. SharePoint’s search is built on top of two different query technologies, with a common index and content crawler that can work across all your document stores. Queries are built using either KQL, the Keyword Query Language, or FQL, the FAST Query Language that was developed for the FAST enterprise search platform acquired by Microsoft in 2008.

Searches can be constructed and run inside web content or through a REST API that connects to the SharePoint search service. This takes both POST and GET requests, simplifying development. It’s simple enough to use this to programmatically deliver queries to SharePoint, using either URLs via GET or as JSON parameters via POST.

In practice, it’s best to use POST and JSON, as this gets around URL length restrictions and supports more complex query types. Both GET and POST return XML-formatted responses by default, though if you prefer, you can choose to receive JSON documents.

As SharePoint is a document store, queries are relatively simple. We’re not looking to return a specific entry; instead, we’re bringing back a list of documents that a user can then click through to explore.

Building REST searches

The Search API takes a familiar form, similar to that used by most Microsoft Graph APIs, though here it’s targeted at your own SharePoint server instances. Starting with a server domain name, it’s a relatively simple structure:

<pre class=”prettyprint”>GET http://server/_api/search/query?query_parameter</pre>

In the example, replace <code>server with the fully qualified domain name of your SharePoint server, with the search terms and any modifiers delivered as the query parameters. This makes a query very easy to construct in your choice of programming languages. GET requests like this are clearly best suited for relatively simple queries. More complex requests can be assembled as JSON documents containing the appropriate key/value pairs for each request element.

Responses can be limited to a set number of documents, or you can choose to make response data paged, loading blocks of results as users move from page to page or adding new results as they scroll through a single-page web application. Paged results like this fit well with providing ready-to-use data for formatting to work with fixed windows in desktop applications.

Although the APIs have their own syntax, you still need to think about KQL and FQL when constructing your query terms. For example, if you want to refine responses, then you’ll need FQL filters. These let you choose the documents returned in your result set. Using the GET API limits you to one filter. For more, use POST to deliver an array of FQL filters to your search server.

Customizing results

Unlike most search environments SharePoint’s API gives you a lot of options to customize results. That’s not surprising—this isn’t a general-purpose search tool. It needs more control, for example, ensuring that roles and other access control mechanisms are applied, as well as giving your SharePoint admins the ability to apply some level of editorial control to results to set how results are ordered and displayed. SharePoint queries are, by default, only available to authorized users, but you can allow a limited set of results for anonymous users, a useful feature if you’re powering public document stores off SharePoint-based web servers.

As well as the search REST API, Microsoft offers a .NET SharePoint set of tools that supports search and a related JavaScript library. This gives you access to the same set of features but makes it easier to work with native response objects rather than parsing JSON or XML. You’ll find both FML and KML operators useful for refining queries, as SQL is not supported. Features such as NEAR help find related content, and XRANK helps manage the response ranks.

Suggested queries

One useful feature is the option to return suggested queries based on what other users have searched for. The SharePoint search API offers a simple Suggest endpoint for this, using parameters to deliver possible search terms that can trigger a suggestion. These are likely to be your user’s current search terms to help them quickly refine queries either before or after running a search. Your request can include the number of suggestions to include and add formatting rules to ensure responses are capitalized correctly.

You could use this approach with a free text input to build your own equivalent of major search engines’ dynamic query builder by requesting suggestions each time a user enters some text. However, in practice, this can be overly complex, and it’s easier to deliver a set of “other users looked for” suggestions after returning a response.

Other options will let your suggested responses include names, which allows you to surface individuals related to search terms. This can help show subject matter experts or members of relevant teams, enabling users to focus searches more effectively.

Using SharePoint as your enterprise document repository continues to make sense, and Microsoft is enhancing it with regular updates for both on-premises and in-cloud instances. Having relatively simple APIs like its search tool helps, as you can quickly build your own custom interfaces to SharePoint-hosted content on the web, on mobile, or on PCs.