Using FQL to query FAST Search Index

In SharePoint 2010, you can use FAST Query Language (FQL) to query the FAST Search Server content index. In this blog, I will show how you can use the Query Web Service and the KeywordQuery class to execute an FQL query against the content index.

You need to ensure the following has been completed, before you can use FQL to search the content index:

·FAST Search Server 2010 for SharePoint has been installed and configured. Here is a good TechNet resource.

·Managed properties have been created and configured. Here is a good MSDN blog post.

·A full crawl has been run on the content source. Here is a good TechNet resource.

 

I will explain the usage of FQL in the context of a Visual Web Part, which can be created using Visual Studio 2010. The code sample below uses FQL to query the content index, and then binds the results to an ASP.NET Repeater control for display purposes. You can also use an ASP.NET GridView control to display the results.

Using FQL with Query Web Service

1.Add a web reference to the search query web service in your Visual Studio project. The path to the web service is similar to the following. Replace Server_Name, sites, Site_Name with the appropriate values in your environment.
http://Server_Name/[sites/][Site_Name/]_vti_bin/search.asmx

 

2.Create an instance of the web service proxy class and set it to use default credentials as shown in the code below.

// Instantiate the Web service proxy
QueryWebServiceProxy.QueryService queryService = new QueryWebServiceProxy.QueryService();

 

// Use default credentials

queryService.Credentials = System.Net.CredentialCache.DefaultCredentials;

 

3.Build the FQL query and run the QueryEx() method. This returns the results in an ADO.NET DataSet object. In my example, I am building the FQL query in a separate function called, BuildQueryXML. The code for this function is provided in step 4.

 

// Build FQL query and execute the QueryEx method, which returns the results in a DataSet

string strQueryXML = BuildQueryXML();

DataSet queryResults = queryService.QueryEx(strQueryXML);

 

4.The BuildQueryXML() function code is shown below. In the code below, I am getting the current date and converting it to yyyy-MM-dd format. Using the StringBuilder object, I am then building an FQL query to get items that are based on the News ‘Content Type’, and whose ‘Publishing Date’ is less than or equal to Today, and whose ‘Featured’ value is set to true. The results are sorted by Publishing Date in descending order. Content Type (‘text’ value), Publishing Date (‘date’ value) and Featured (‘boolean’ value) are managed properties in this case. After that, I am building the complete Query Packet with the FQL query. And finally, the function returns the Query Packet generated as a string value. I have hard-coded the number of results returned to 5. You can change it to make it more dynamic, by providing a custom property in the web part.

NOTE: There is an excellent tool on CodePlex called, MOSSMAN’s FAST for SharePoint Query Tool, which greatly simplifies FQL query construction. I highly recommend using this tool to build your FQL queries. In a nutshell, it allows you to pick the managed properties that you want to query, generates the FQL query, and runs it against the content index. You can then copy the generated Query Packet (with FQL) and add it to your code and make necessary edits as needed.

private string BuildQueryXML()

{

// Get today’s date and convert it to 2011-05-01 format
// (i.e. 4 digit year – 2 digit month – 2 digit day)

DateTime dtToday = System.DateTime.Today;

string strToday = dtToday.ToString(“yyyy-MM-dd”);

// Build the FQL query here
StringBuilder strFQLQuery = new StringBuilder(@”and(filter(contenttype:equals(“”News””)), filter(featured:equals(“”true””)), filter(publishingdate:range(min,datetime(“””);

strFQLQuery.Append(strToday);

strFQLQuery.Append(@”””), to=””LE””)))”);

// Build the complete Query Packet here with the FQL query
StringBuilder xmlString = new StringBuilder(@”<QueryPacket Revision=””1000″”>

<Query>

<Context>

<QueryText language=””en-US”” type=””FQL””>”);

 

xmlString.Append(strFQLQuery);

xmlString.Append(@”</QueryText>

</Context>

<SupportedFormats Format=””urn:Microsoft.Search.Response.Document.Document”” /><ResultProvider>FASTSearch</ResultProvider>
<Range>

<StartAt>1</StartAt>

<Count>5</Count></Range>

<EnableStemming>true</EnableStemming>`<EnableSpellCheck>Suggest</EnableSpellCheck><IncludeSpecialTermsResults>true</IncludeSpecialTermsResults><IncludeRelevantResults>true</IncludeRelevantResults><ImplicitAndBehavior>true</ImplicitAndBehavior><SortByProperties>
<SortByProperty name=””publishingdate”” direction=””Descending”” />

</SortByProperties> <TrimDuplicates>true</TrimDuplicates><Properties>
<Property name=””Title”” />
<Property name=””publishingdate”” />

</Properties>
</Query>
/QueryPacket>”);

//convert query to string format and return

return xmlString.ToString();

 

}

 

5.The DataSet object returned by the QueryEx() method contains a DataTable object for each search results set returned for the specified query. In the code below, I am retrieving a DataTable called RelevantResults. This contains search results from the content index that matches the query. This table contains one row for each search result, and one column for each requested managed property. I am also creating a DataView object using the RelevantResults DataTable, and then finally bind the DataView to the Repeater control.

 

DataTable dtNewsItems = queryResults.Tables[“RelevantResults”];

 

// Store the DataTable obj containing the news items results in a DataView object

DataView dvNewsItems = new DataView(dtNewsItems);

 

// Bind the DataView object containing the results to the Repeater control

rptNewsItems.DataSource = dvNewsItems;

rptNewsItems.DataBind();

 

 

Using FQL with KeywordQuery class

1.Get a handle to the SPServiceContext object related to the current site

// get a handle to the service context of the current site

SPServiceContext serviceContext = SPServiceContext.GetContext(SPContext.Current.Site);

 

2.Create an instance of the SearchServiceApplicationProxy class

 

// create a proxy for the SP search service

SearchServiceApplicationProxy searchProxy = (SearchServiceApplicationProxy)SearchServiceApplicationProxy.GetProxy(serviceContext);

 

3.Create an instance of KeywordQuery object and assign relevant properties. Also, call a function, BuildFQLQuery() to build the FQL query. This function is similar to the BuildQueryXML() function in described in step 4 of the earlier section, but differs in one aspect – it does not build the QueryPacket xml. The KeywordQuery object creates the Query Packet. All we need to do is to provide the FQL query.

 

// instantiate KeywordQuery instance

KeywordQuery keywordQuery = new KeywordQuery(searchProxy);

 

 

//Set KeywordQuery to use FQL

keywordQuery.EnableFQL = true;

 

//Set Results Provided to FASTSearch

keywordQuery.ResultsProvider = SearchProvider.FASTSearch;

 

//get first row

keywordQuery.StartRow = 1;

 

//set the number of rows returned in search results

keywordQuery.RowLimit = 5;

 

// sort results by publishingdate in descending order

keywordQuery.SortList.Add(“publishingdate”, Microsoft.Office.Server.Search.Query.SortDirection.Descending);

 

//specify properties to return in search results

keywordQuery.SelectProperties.Add(“Title”);

keywordQuery.SelectProperties.Add(“publishingdate”);

 

keywordQuery.ResultTypes = ResultType.RelevantResults;

// call function to build FQL query
strFQLQuery = BuildFQLQuery();

 

//set the FQL query here

keywordQuery.QueryText = strFQLQuery;

 

// execute the query, and store the results in ResultsTableCollection

ResultTableCollection results = keywordQuery.Execute();

 

//store relevant results in Results Table object

ResultTable relevantResults = results[ResultType.RelevantResults];

 

DataTable dtNewsItems = new DataTable(“NewsItems”);

dtNewsItems.Load(relevantResults, LoadOption.OverwriteChanges);

 

 

//Store the DataTable obj containing the news items results in a DataView object

DataView dvNewsItems = new DataView(dtNewsItems);

 

// Bind the DataView object containing the results to the Repeater control

rptNewsItems.DataSource = dvNewsItems;

rptNewsItems.DataBind();

 

4.The BuildFQLQuery() function code is shown below. Note that we are not building the entire Query Packet xml here.

private string BuildFQLQuery()

{

// Get today’s date and convert it to 2011-05-01 format
// (i.e. 4 digit year – 2 digit month – 2 digit day)

DateTime dtToday = System.DateTime.Today;

string strToday = dtToday.ToString(“yyyy-MM-dd”);

// Build the FQL query here
StringBuilder strFQLQuery = new StringBuilder(@”and(filter(contenttype:equals(“”News””)), filter(featured:equals(“”true””)), filter(publishingdate:range(min,datetime(“””);

strFQLQuery.Append(strToday);

strFQLQuery.Append(@”””), to=””LE””)))”);

// convert search query to string format and return

return strFQLQuery.ToString();

 

}

 

Benefits of using FQL to query FAST Search Index

1.No need to write complex CAML queries to query multiple lists in multiple web sites.

2.The FAST Search crawler does all the heavy lifting for you, and builds the content index. And you then simply query the content index to retrieve results.

3.All security trimming is performed as part of the query matching, based on ACL information that is stored in the search index. Only items that the user has access to are returned in the search results.