Thursday, August 9, 2018

Why Sitecore Recommends SOLR For Site Search?


Before we understand what SOLR is and why Sitecore recommends it for search, let us understand some basics regarding different type of databases we have.

SQL
NoSQL
Table based databases
Document based databases
Have predefined schema
Dynamic schema
Vertically scalable. Manage increasing load by increasing CPU, RAM etc. on single server.
Horizontally scalable. We can add few more servers easily to handle large traffic.
Good fit for complex queries.
Not good fit for complex queries.
Based on ACID properties (Atomicity, Consistency, Isolation and Durability)
Based on Brewers CAP theorem (Consistency, Availability and partition tolerance)
Good fit for transactional type applications.
Not stable enough for complex transactional applications.
MS-SQL, Oracle, MySQL, SQLite, Postgres
MongoDB, Redis, BigTable, RavenDb, Cassandra, Hbase, Neo4j and CouchDb


Full text search of SQL:

Pros: For smaller databases where we have limited content and fixed schema, full text search of SQL is ideal.

Cons: But since SQL is not horizontally scalable, performing search with limited resources (CPU, RAM etc.) can be a huge set back to the application.

Full text search with NOSQL:  

Pros: NoSQL should be used when Schema (data requirements) isn’t clear at the outset or if we are dealing with massive amounts of unstructured data. Think of non-relational databases more like folders, assembling related information of all types. For example, if a blog application used a NoSQL database, each file could store data for a blog post: social likes, photos, text, metrics, links etc. 

Cons: But storing and retrieving data in bulk like this requires extra processing effort and more storage than highly organized SQL data.

Full Text Search Engines:

Full text search engines excel at quickly and effectively searching large volumes of unstructured text and returning these documents based on how well they match the user’s query. They also have the ability to quickly facet, or categorize data or search results based on specific values of specific fields.

Example: Lucene, Solr, ElasticSearch, KinoSearch, Sphinix, Xapian.

Lucene: Lucene is able to achieve fast search responses because, instead of searching the text directly, it searched an index instead.

This is like retrieving pages in a book related to a keyword by scanning the index at the back of a book, as opposed to searching every word of every page of the book.

Advantages of Lucene:

  1. Indexing: Indexing is the process of crawling through content and storing them on disk. Most of the search engines only support batch indexing; once they create an index for a set of documents, adding new documents becomes difficult without reindexing all the documents. Lucene supports both Incremental and batch indexing. Lucene allows easy adding of documents to an existing index.
  2. Data Sources: Many search engines can only index files or webpages. Lucene in addition can index data from a database, or where multiple virtual documents exist in a single file, such as a ZIP archive.
  3. Content Tagging: Lucene supports content tagging by treating documents as collection of fields, and supports queries that specify which fields to search.
  4. Stemming: Reducing a word to its root form is called Stemming. Often, a user desires a query for one word to match other similar words. For example, a query for “jump” should probably also match words “jumped”, “jumper”, or “jumps”.
  5. Stop-word processing: Common words, such as “a”, “and”, “the” etc. add little value to a search index. Lucene uses StopAnalyzer class which eliminates stop words from the input stream.
  6. Query features: Lucene supports full Boolean queries, and queries and also has ability to search multiple indexes at once and merge the results to give a meaningful relevance score.
  7. Concurrency: Lucene allows users to search an index transactionally, even if another user is simultaneously updating the index.
  8. Non-English support: Lucene allows us to perform language-specific filtering.

Disadvantages of Lucene:

  1. Scaling: Lucene works perfectly fine for single server environment. But in multi-server environment indexes have to be copied to each server which is error prone.
  2. NO UI: We have to depend on external tools like LUKE to analyse indexes and run queries during development.





Lucene Vs Solr

A simple way to conceptualize the relationship between Solr and Lucene is that of a car and its engine. You can’t drive an engine, but you can drive a car. Similarly, Lucene is a programming library which you can’t use as-is, whereas Solr is a complete application which can be used out-of-box.

Solr = Lucene + additional features

Solr is an application or HTTP wrapper on top of Lucene. Content from the Lucene index can be retrieved using an HTTP GET query in XML, JSON, or binary formats.

Extra features in addition to Lucene
  1. Faceted Search: Dynamically clusters search results into drill-down categories.
  2. Built-in Sorting: Automatic features to sort search results by a variety of characteristics.
  3. Web Admin Interface: SOLR application provides us with a web UI to run queries and check indexed data in various formats. We need not depend on external tools like LUKE.
  4. Hit Highlighting: Shows a snippet of a document in the search results that surrounds the search terms.
  5. HTTP query: Pass a number of optional request parameters to the request handler to control what information is returned.


Conclusion: Sitecore strongly recommends going with SOLR when we have scaled environment.  Lucene can only be used for a development or single server environment.





Friday, December 5, 2014

Sitecore Custom Validation Example

Following example is used to create an Even number validator using sitecore

Step 1: Create a new class inherited from StandardValidator class

using Sitecore.Data.Validators;
using System.Runtime.Serialization;
namespace Demo.Validators
{
    [Serializable]
    public class EvenNumberValidator : Sitecore.Data.Validators.StandardValidator 
    {
        public EvenNumberValidator() { }
        public EvenNumberValidator( SerializationInfo info,StreamingContext context):base(info,context)
        {
        }
    }
}

Step 2: Override Name property and Evaluate & GetMaxValidatorResult methods.

using Sitecore.Data.Validators;
using System.Runtime.Serialization;
namespace Demo.Validators
{
    [Serializable]
    public class EvenNumberValidator : Sitecore.Data.Validators.StandardValidator 
    {
        public EvenNumberValidator() { }
        public EvenNumberValidator( SerializationInfo info,StreamingContext context):base(info,context)
        {

        }
        public override string Name
        {
            get {
                    return (this.ToString()); // returns validator name to be displayed in error message
                }
        }

        protected override ValidatorResult Evaluate() //executes on validation

        {
            string val = GetControlValidationValue(); //fetches value of a control. GetItem() can be used to fetch complete item
            int n;
            bool isEven = false;
            bool isNumeric = int.TryParse(val, out n);
            if(isNumeric)
            {
                isEven = n % 2 == 0 ? true : false;
            }
            if(isEven)
            {
                return (ValidatorResult.Valid); // validation success
            }
            else
            {
                //validation failed
                Text = String.Format("{0} is not an even number!!",val); //Error message to be displayed
                return (GetFailedResult(ValidatorResult.Error)); //error level
            }
        }
        protected override ValidatorResult GetMaxValidatorResult()//blocks operations if we return FatalError or CriticalError
        {
            return (GetFailedResult(ValidatorResult.Error));
        }
       
    }
}

If the result of the GetMaxValidatorResult() method is FatalError or CriticalError, then Sitecore will block operations such as save or workflow commands in the user interface until validation completes.
Validation Bar and Quick Action Bar validators never block user interface operations.

Step 3: Create a new validator (here Even Number Validation)  at /sitecore/system/Settings/Validation Rules/Field Rules section using /sitecore/templates/System/Validation/Validation Rule template. Set Title, Description and Type properties.





























Step 4:  Assign the validator to any field in a template. Here I have assigned it to “Validate Button” type of validation.














Step 5: Perform validation using “Validation” button under “Review” tab.

Thursday, December 4, 2014

Sitecore Validation Examples

For each item, Sitecore invokes the item validation rules defined in the item or the standard values item associated with its data template, as well as global validation rules.

For each data template field, Sitecore invokes the validation rules defined in the data template field definition item, as well as the validation rules defined in the data template field type validation rules definition item.

After you stop editing a field value, Sitecore automatically invokes validation rules asynchronously, and then updates the user interface after validation completes.

Validators.UpdateDelay property in web.config controls the length of time between editing activity and the invocation of validators.

Validators.AutomaticUpdate property when set to false would disable automatic validation feature.

Validation Levels

Field Level: We can configure validators to be used for validation at the individual field level. For example, required field validator for a single line text field.  All field level validators are present at /sitecore/system/Settings/Validation Rules/Field Rules.

Field Type Level: We can configure validators to be used for validation of all occurrences of a particular field type. For example, all occurrences of DropList field type in sitecore tree. All field type level validators are present at /sitecore/system/Settings/Validation Rules/Field Types

Item Level: We can configure validators for individual items in its presentation details. All item level validators are present at /sitecore/system/Settings/Validation Rules/Item Rules

Standard Values Level: Configure validation for all items based on a data template by configuring validation rules in the standard values of that data template.

Global Level: Configure validation that applies to all items.

Sitecore Validation can be done in four different ways
  1. Quick Action Bar.
  2. Validator Bar.
  3. User chooses Validate command in the Proofing group on the Review tab.
  4. User chooses a specific workflow command.

     1. Quick Action Bar

     Steps to display field validation error messages on Quick Action Bar

     Step 1: Add validator for individual field in template in Quick Action Bar Section.
     Step 2: Right click on Quick Action Bar and enable “Validation Rules”

       



































      Step 3: Creating new page would display validation error on Quick Action Bar if “Page Title” value is empty.




       Note: We can save and publish the page even with validation errors.

     2. User chooses Validate command in the Proofing group on the Review tab.

    Step 1: Add validator for individual field in template in Validate Button Section.













    Step 2: Click on “Validation” button under “Review” tab without entering “Page Title” value.

























     3. Validation Bar
     Steps to display error message on Validation Bar

    Step 1: Add validator for individual field in template in Validator Bar Section.












   Step 2: Select the page and click on "Save" button without entering "Page Title" value.




















    4. Workflow

    Step 1: Create a workflow definition item “PageReview” in “/system/Workflows” section using “/sitecore/templates/System/Workflow/Workflow” template.






















   Step 2: Create three workflow states “Draft”, “AwaitingApproval” & “Approved” using “System/Workflow/State” template under “PageReview” workflow definition item.






















  
     Step 3:  Select workflow definition item and set the Initial State field to Draft.






















    Step 4: Create “Submit” command under “Draft” state using template “/System/Workflow/Command”






    Step 5: Create “Approve” and “Reject” commands under “AwaitingApproval” state using template “/System/Workflow/Command”.























    Step 6: Set “Next State” of “Approve” command to “Approved” state and “Reject” command to “Draft” state and “Submit” command to “AwaitingApproval”.





















  Step 7: Add “Validation Action” to the “Approve” command using “System/Workflow/Validation Action” template.




















    Step 8: In the type field, enter the following
   Sitecore.Workflows.Simple.ValidatorsAction, Sitecore.kernel
   In Max Result Allowed field enter “Warning”
   Fill Error result fields with message “You cannot approve an item with validation errors




























   Step 9: Define the Final state to “Approved” state.


















   Step 10: Assign the workflow to page template. Select “_Standard Values”, click on “View” tab in main menu and select “Standard Fields”



























   Assign “Default workflow” to the “PageReview” workflow item.


   


   Step 11: Assign “Required” validator to “Page Title” field













   Step 12: Create a new page based on the template. New page would show a workflow warning at the top.
















  Step 13: Click on “Submit” action under “Review” tab.

















   Step 14: Click on “Approve” button to trigger work flow validation modal.



Monday, September 29, 2014

Sitecore Proxies And Clones

A)  Proxies:

Proxy items duplicate’s an item (or sub tree) from one part of the content tree to another. Any Update to proxy item will update the source item and vice versa.


Proxies are generally used when we need an item appear as a child item to multiple parent items. Following example demonstrates a book item appearing under multiple genre items.























Step 1: Set proxiesEnabled element to true in web.config at /configuration/sitecore/databases for master and web databases.

<databases> 
<database id="master" singleInstance="true" type="Sitecore.Data.Database, Sitecore.Kernel">    
<proxiesEnabled>true</proxiesEnabled>   
</database>
</databases>

Step 2: Create two new templates “BookCategory” & “Book” with “Title” & “Description” fields.


















Step 3: Create book items from book template in a common folder.


















Step 4: Create “Comedy”, “Fiction” & “Romance” categories from “BookCategory” template.





























Step 5: Create proxy items at /Sitecore/System/Proxies location using /System/Proxy data template.


Step 6: Set the source item and target item of proxy. Here, “Book1” is the source item and “Comedy” is the target item.






















Step 7: Create other proxies as required.





















Step 8: Editing any proxy item would affect source item and vice versa

Publishing a proxy
<publishVirtualItems> setting in web.config indicates whether Sitecore should publish virtual items as if they were normal items.
If this setting is set to true, virtual item becomes a real item and the relationship between virtual item and parent item no longer exist.
If this setting is set to false (preferred), relationship between virtual item and parent item exist even on web database if proxies are enabled on the web database.

B)  CLONES

Cloning creates a dynamic duplicate of an item or branch in the content tree to another different location within the same content tree. A clone inherits field values from the parent item instead of standard values and provides notification to clone item whenever parent item is modified.

If all fields of parent item and cloned item are same, then any change to parent item would apply changes to all cloned items immediately without any prompt.

If the value in the cloned item is different to parent item, Sitecore creates a notification, which the user must accept in order to apply the change to the cloned item.

Cloned items when published, become real items and the relationship between cloned item and parent item no loner exist.

Following steps show an example of creating a clone

Step 1: Select the parent item to be duplicated and click on “Clone” button in the “Configure” section.




















Step 2: Select the destination for the cloned item and click on “Clone” button.




















Step 3: Changing field value in the cloned item will not affect the parent item.










































Step 4: Change parent item field value. Since content of parent and cloned item are different, sitecore creates a notification which user must accept.























UNCLONE: A clone can be converted to a normal item, severing any ties it has to the original and become completely independent by uncloning.