Monday, 4 February 2008

MOSS 2007 Custom Search

Out of the box search in MOSS 2007 is very powerful, what if you have very specific requirements? Use the classes in the Microsoft.Office.Server.Search.Query namespace in a custom Web Part or User Control and add it to your SharePoint page layout. You may have specific requirements for rendering or specific search filters that are not provided by the out of the box search.
A small suggestion : Try your best to use the Out of the Box Search, you will save alot of time, and it is very powerful and comes with alot of goodies.
I have previously used SharePoint Portal Server 2003 Search Web Service to provide website search functionality in a CMS 2002 web application. Search gets complicated when you need to integrate searching across different content sources, there was a CMS / SharePoint connector that allowed you to register Meta properties on CMS pages, sourced from an external system (a database containing Tourism data, rendered in web pages), you then use SharePoint to crawl your CMS site and the Meta propeties are also indexed and you can then use them to provide more advanced search capabilities.
Microsoft Office SharePoint Server 2007 makes that whole process alot easier, using the BDC. If you want to provide search across different systems, then MOSS can be used to index that data, so you can search it.

1. Create the BDC Application Definition file, and import it into your SSP.
In order for Search to 'Crawl' your BDC application, the application definition must contain a ProductIDEnumerator method. This method is used by search to return ALL the Identifiers (primary key, unique record identifier) for all records in the backend system. SharePoint then loops through all the IDs and returns a detailed record for each ID. Each record is then indexed and can then be used in search.
On importing your application definition, profile pages are created for your BDC Entities, browse to these pages and see if you can get data to display, if it does, then crawling the data in your external system may work.

2. Create a Business Data Content Source in Search Settings
Use your Shared Services Provider. Your SharePoint farm must have an SSP for search to work. When creating the Business Data Content Source, select your BDC application as the source. Then run a full crawl.

3. Populate Search using the Content Source
At this stage the items in the search index will increment to the total number of items that can be read from your external system. If that is not happening, you may be getting errors in the crawl process, fix them. (easier said than done, this can be a painful, time wasting process).
On the first crawl of your BDC application in search, the various fields you have defined will be detected as crawled properties.

4. Create the Managed Properties and Map them to Crawled Propeties
"... Mapping Crawled Properties to Managed Properties
To make a crawled property available for the Search experience, you must map it to a managed property. Mapping the crawled properties to managed properties makes them available for Search queries and makes them appear in Advanced Search and search results. You can map multiple crawled properties to a single managed property or map a single crawled property to multiple managed properties. If a managed property has multiple crawled properties mapped to it, and a document contains values for more than one of the crawled properties, the order in which the properties are mapped and their priority determine the managed property’s value... "
In search settings, click on Meta Property Mappings then, on the left hand menu you can switch between viewing managed properties and crawled properties.
So go through and create Managed Properties, and associate them to crawled properties detected when crawling your BDC. What's cool is that DataTypes are recgonised ! So if you define an integer or boolean or datetime as an attribute of a BDC entity, you create a managed property of that type. A cool feature is the ability to map more than one crawled property to a managed property.

5. Re-Crawl
You need to re-crawl your BDC content source now, as after associating the crawled properties to managed properties, a fresh crawl is required to populate them in the search index. If you don't re-crawl their won't be any items in the properties, and if the properies are used in search queries, it won't work.
When you view individual managed properites, the number of items in the search index which use them is listed on the page, so after crawling, check your newly created property and it should be populated.

6. Search has indexed the data, time for some programming
Get the MOSS 2007 SDK, or browse to Creating Custom Enterprise Search Applications for the online version.
The SDK states the three different Search Query types that can be used.
Enterprise Search in Microsoft Office SharePoint Server 2007 supports three types of search syntax for building search queries:
Keyword Query syntax (search terms are passed directly to the Enterprise Search service)
SQL syntax (extension of SQL syntax for querying databases)
URL syntax (search parameters are encoded in URL, and posted directly to the search page)
I have decided to go with the second option (SQL Syntax) the Enterprise Search SQL Query Language is a SQL style language used to build search queries. I say SQL style, coz it aint regular SQL! Also, I have used SharePoint 2003 SPS Query Language, and I found this to be fairly similar.
What is cool about building SQL Syntax search queries is you can use those Managed Properties mapped to crawled BDC properties, so that you can create an "Advanced Search".
I have done this, and for example, created a search User Control on "Accommodation" data.
In my custom search you can search for accommodation in Western Australia, and specify :
Type : Hotel, Motel, Resort, Cabin etc.
Price Range : $100 - $200
Disabled Access
Are Pets Allowed
Are Children Catered For
In a particular town : Perth
In a particular region : Australia's South West
With facilities : Bar, Fridge, Air Conditioning, Internet acess etc

We also needed Search across Event data, this required alternate criteria to be specified.
The Type of Event
Specify a Start Date Range for Events
Specify an End Date Range for Events
etc.
or for Tours, users need to search for
Towns Tour Visits
etc.
So I built a User Control which allows a user to specify combinations of the above criteria. The dude I sit next to at work was working on an awesome Suggest style ASP.NET User Control, it looks cool, I needed to use that control, and I was able to, by placing it in my User Control. I then placed that control in a MOSS 2007 Page Layout.
After the search is triggered and rendered users can then :
Page through results
Change Results Per Page
Re-Sort again based on Title, Price, Location etc. or by Date for Events
Click another link to filter results to a specific category

How did I provide all that filtering, and ordering functionality ?
By using the Search Managed Properties, which were mapped to BDC Crawled Properties in my Enterprise Search SQL Query.
The WHERE and ORDER BY clauses work similar to SQL, you can filter and order your results using data from the Meta Property Mappings.
For example, a the following query includes Managed Properties which are mapped to BDC properties, The query uses those BDC properties to both restrict the search to a particular category and domestic region, and also uses another BDC property to order the results first by the products' membership level, then by Rank (Pay us more $ and we'll rank you higher?)

SELECT Title, Path, Description, HitHighlightedSummary, Rank, ProductMembershipRank, DomesticRegion, ProductCategory
FROM SCOPE()
WHERE FREETEXT(Description,'Fun') AND ProductCategory = 'Accommodation'
AND DomesticRegion = 'Australia''s South West'
ORDER BY MembershipRank DESC, RANK DESC

The code needed in your custom web part or user control to execute the search query is :

FullTextSqlQuery ftsq = new FullTextSqlQuery(ServerContext.Current);
ftsq.ResultTypes = ResultType.RelevantResults;ftsq.QueryText = ;
//Return the search results to a ResultTableCollection
ResultTableCollection results = ftsq.Execute()
You can get a DataSet of results from the ResultTableCollection of search results. Then render this DataSet or get the XML representation and render that.
There are also other useful data you can get from ResultTableCollection such as ElapsedTime so you can also get the time taken to execute the search and render that to the user. The HitHighlightedSummary is very cool, it returns fragments of the content in a search result, and the term searched for can be highlighted so you can render search results that look cool, like a real search engine.
I created a "wrapper" class around search. it has an execute method, and alot of properties which you assign filter values to. My Execute method calls other private methods which build the Search SQL Query based on the properties set on the object and then use the FullTextSqlQuery class. but it also has Query properties, such as SelectProperties, WhereClause, OrderByClause which are created in the process.
So I can "chop up" and build search queries from existing ones, and do other cool OO stuff. it takes some time to code (any fans of dynamic sql generation code out there?) a wrapper class, but some investment like that is worth it, and allows other developers you work with to use Search to get data a little easier, without having to mess with queries if they don't want to.
One very cool feature you can provide in search, if your backend system entities have specified a URL path to a thumbnail image, you can use that to render image thumbnails in search results, looks very cool. That was actually a requirement for rendering which I had to provide, and could not using the OOTB search, also the requirements were very picky on searching across specific columns of data and not others, as well as specific weightings for columns, by generating queries myself I have full control.

It all depends on your data and requirements
So should you go Out-Of-The-Box or Do It Yourself for search?
Depends on whether or not you need to provide many advanced search filters and have very specific requirments on the rendering of the search results. You also have the control of building the queries and getting the data back from search and doing what you want with it. If your backend system requires very advanced specifc fine control searching, then build your own search queries and take more control over the search process. DIY also lets you use AJAX style search controls, and do pretty much anything else you want to because you have full control over the rendering of the Search Criteria UI and the Search Results.

1 comment:

Josh Noble said...

If the out of the box search is insufficient, or you don't want to spend weeks reprogramming MOSS Search, there are always web parts like Ontolica which can add functionality like wildcard and faceted search.