Introduction to Lucene and Solr - 2

•Download as PPTX, PDF•

3 likes•898 views

YI-CHING WU

Software

Preparation
 NB/PC or VM
 Install JRE
 Install Tomcat on Windows / Linux
 Download Solr 4.6
 Download JDBC : sqljdbc4.jar
 Un-Zip Solr 4.6
2

Install Solr on Windows
3
 install java runtime
 download Tomcat windows service and install it to d:Tomcat
 test Tomcat site http://localhost:8080
 stop Tomcat
 download Solr and Unzip to d:tempsolr
 copy d:tempsolrdistsolr-x.x.x.war to d:tomcat
 rename d:tomcatsolr-x.x.x.war to d:tomcatsolr.war
 copy d:tempsolrexamplesolr to d:websolr
 add Java Options -Dsolr.solr.home=d:websolr
 add Java Options -Dsolr.velocity.enabled=false
 copy d:tempsolrexamplelibext to d:tomcatwebappssolrWEB-INFlib
 created:tomcatwebappssolrWEB-INFclasses
 copy d:tempsolrexampleresourcelog4j.properties to d:tomcatwebappsso
lrWEB-INFclasses
 start Tomcat
 visit http://localhost:8080/solr/

Setup Schema.xml
 Data source overview
 Edit the schema.xml
4

5
Import Data to Solr
- Data import Handler  Conntect to “TW-TSDev01”
 Add new “/dataimport” request handler
 Edit “data.import.xml"
 Command
 http://localhost:8080/solr/core0/dataimport?command=full-import&
commit=ture
 http://localhost:8080/solr/core0/dataimport?command=delta-import&
clean=false&commit=ture (dataimport.properties)
 http://wiki.apache.org/solr/DataImportHandler

Scheduling and
dataimport.properties.xml
7
 /lib/apache-solr-dataimportscheduler.
jar
 Tomcat : /solr.war
<listener><listener-class>
org.apache.solr.handler.dataimport.sc
heduler.ApplicationListener
</listener-class></listener>

Query data
- Solr administration Console
 Analysis
 DataImport
 Try to query the data we feed to Solr
8

Advance Search- Edismax
9
 &defType=Edismax
 select/?q=video&defType=edismax&qf=features^20.0+text^0.3&bq=ca
t:electronics^5.0

Full example
q=osce&start=0&rows=10&wt=xml&indent=false
&defType=edismax
&hl=true&hl.fl=Title%2CSegment%2CProblems%2CPubli
cContent
&hl.simple.pre=%3Cem%3E&hl.simple.post=%3C%2Fem
%3E&hl.usePhraseHighlighter=true&hl.highlightMultiTer
m=true
&facet=true&facet.query=osce
&facet.field=Segment,OperatingSystem
&spellcheck=true&spellcheck.build=true&spellcheck.r
eload=true
&spellcheck.q=oscc
&spellcheck.onlyMorePopular=true
&spellcheck.accuracy=0.7
10

Setup solrconfig.xml
 Spell checking
 Suggestion
 List Terms of field
 MLT – more like this
11

SolrJ
 Library to connect and interact with Solr
12

Viewers also liked

Lucene and MySQLfarhan "Frank" mashraqi

Lucandraotisg

DocValues aka. Column Stride Fields in Lucene 4.0 - By Willnauer Simonlucenerevolution

The Evolution of Lucene & Solr Numerics from Strings to Points: Presented by ...Lucidworks

Building and Running Solr-as-a-Service: Presented by Shai Erera, IBMLucidworks

Lucene for Solr DevelopersErik Hatcher

Berlin Buzzwords 2013 - How does lucene store your data?Adrien Grand

Near Real Time Indexing: Presented by Umesh Prasad & Thejus V M, FlipkartLucidworks

Architecture and Implementation of Apache Lucene: Marter's ThesisJosiane Gamgo

Lucene Introductionotisg

Text categorization with Lucene and SolrTommaso Teofili

Lucene basicsNitin Pande

Introduction to Lucene & Solr and UsecasesRahul Jain

Apache Lucene: Searching the Web and Everything Else (Jazoon07)dnaber

Introduction to Elasticsearch with basics of LuceneRahul Jain

Integrating Hadoop & SolrLucidworks (Archived)

Dawid Weiss- Finite state automata in luceneLucidworks (Archived)

Building a real time, solr-powered recommendation engineTrey Grainger

What is in a Lucene index?lucenerevolution

Building a distributed search system with Hadoop and LuceneMirko Calvaresi

Viewers also liked (20)

Lucene and MySQL

Lucandra

DocValues aka. Column Stride Fields in Lucene 4.0 - By Willnauer Simon

The Evolution of Lucene & Solr Numerics from Strings to Points: Presented by ...

Building and Running Solr-as-a-Service: Presented by Shai Erera, IBM

Lucene for Solr Developers

Berlin Buzzwords 2013 - How does lucene store your data?

Near Real Time Indexing: Presented by Umesh Prasad & Thejus V M, Flipkart

Architecture and Implementation of Apache Lucene: Marter's Thesis

Lucene Introduction

Text categorization with Lucene and Solr

Lucene basics

Introduction to Lucene & Solr and Usecases

Apache Lucene: Searching the Web and Everything Else (Jazoon07)

Introduction to Elasticsearch with basics of Lucene

Integrating Hadoop & Solr

Dawid Weiss- Finite state automata in lucene

Building a real time, solr-powered recommendation engine

What is in a Lucene index?

Building a distributed search system with Hadoop and Lucene

Recently uploaded

Folding Cheat Sheet #4 - fourth in a seriesPhilip Schwarz

Buds n Tech IT Solutions: Top-Notch Web Services in Noidabntitsolutionsrishis

How to Track Employee Performance A Comprehensive Guide.pdfLivetecs LLC

Recruitment Management Software Benefits (Infographic)Hr365.us smith

A healthy diet for your Java application Devoxx France.pdfMarharyta Nedzelska

Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...Natan Silnitsky

Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Angel Borroy López

Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio, Inc.

Unveiling the Future: Sylius 2.0 New FeaturesŁukasz Chruściel

Intelligent Home Wi-Fi Solutions | ThinkPalmSujith Sukumaran

Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024StefanoLambiase

英国UN学位证,北安普顿大学毕业证书1:1制作qr0udbr0

Software Project Health Check: Best Practices and Techniques for Your Product...Velvetech LLC

React Server Component in Next.js by Hanief UtamaHanief Utama

Advantages of Odoo ERP 17 for Your BusinessEnvertis Software Solutions

Best Web Development Agency- Idiosys USA.pdfIdiosysTechnologies1

Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odishasmiwainfosol

What are the key points to focus on before starting to learn ETL Development....kzayra69

KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxTier1 app

CRM Contender Series: HubSpot vs. SalesforceBrainSell Technologies

Recently uploaded (20)

Folding Cheat Sheet #4 - fourth in a series

Buds n Tech IT Solutions: Top-Notch Web Services in Noida

How to Track Employee Performance A Comprehensive Guide.pdf

Recruitment Management Software Benefits (Infographic)

A healthy diet for your Java application Devoxx France.pdf

Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...

Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...

Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data

Unveiling the Future: Sylius 2.0 New Features

Intelligent Home Wi-Fi Solutions | ThinkPalm

Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024

英国UN学位证,北安普顿大学毕业证书1:1制作

Software Project Health Check: Best Practices and Techniques for Your Product...

React Server Component in Next.js by Hanief Utama

Advantages of Odoo ERP 17 for Your Business

Best Web Development Agency- Idiosys USA.pdf

Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha

What are the key points to focus on before starting to learn ETL Development....

KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx

CRM Contender Series: HubSpot vs. Salesforce

Introduction to Lucene and Solr - 2

1. Day 2 – Solr Hands On Core Tech @Trend Micro 吳奕慶YI-CHING WU 1

2. Preparation  NB/PC or VM  Install JRE  Install Tomcat on Windows / Linux  Download Solr 4.6  Download JDBC : sqljdbc4.jar  Un-Zip Solr 4.6 2

3. Install Solr on Windows 3  install java runtime  download Tomcat windows service and install it to d:Tomcat  test Tomcat site http://localhost:8080  stop Tomcat  download Solr and Unzip to d:tempsolr  copy d:tempsolrdistsolr-x.x.x.war to d:tomcat  rename d:tomcatsolr-x.x.x.war to d:tomcatsolr.war  copy d:tempsolrexamplesolr to d:websolr  add Java Options -Dsolr.solr.home=d:websolr  add Java Options -Dsolr.velocity.enabled=false  copy d:tempsolrexamplelibext to d:tomcatwebappssolrWEB-INFlib  created:tomcatwebappssolrWEB-INFclasses  copy d:tempsolrexampleresourcelog4j.properties to d:tomcatwebappsso lrWEB-INFclasses  start Tomcat  visit http://localhost:8080/solr/

4. Setup Schema.xml  Data source overview  Edit the schema.xml 4

5. 5 Import Data to Solr - Data import Handler  Conntect to “TW-TSDev01”  Add new “/dataimport” request handler  Edit “data.import.xml"  Command  http://localhost:8080/solr/core0/dataimport?command=full-import& commit=ture  http://localhost:8080/solr/core0/dataimport?command=delta-import& clean=false&commit=ture (dataimport.properties)  http://wiki.apache.org/solr/DataImportHandler

6. 6 Commands example

7. Scheduling and dataimport.properties.xml 7  /lib/apache-solr-dataimportscheduler. jar  Tomcat : /solr.war <listener><listener-class> org.apache.solr.handler.dataimport.sc heduler.ApplicationListener </listener-class></listener>

8. Query data - Solr administration Console  Analysis  DataImport  Try to query the data we feed to Solr 8

9. Advance Search- Edismax 9  &defType=Edismax  select/?q=video&defType=edismax&qf=features^20.0+text^0.3&bq=ca t:electronics^5.0

10. Full example q=osce&start=0&rows=10&wt=xml&indent=false &defType=edismax &hl=true&hl.fl=Title%2CSegment%2CProblems%2CPubli cContent &hl.simple.pre=%3Cem%3E&hl.simple.post=%3C%2Fem %3E&hl.usePhraseHighlighter=true&hl.highlightMultiTer m=true &facet=true&facet.query=osce &facet.field=Segment,OperatingSystem &spellcheck=true&spellcheck.build=true&spellcheck.r eload=true &spellcheck.q=oscc &spellcheck.onlyMorePopular=true &spellcheck.accuracy=0.7 10

11. Setup solrconfig.xml  Spell checking  Suggestion  List Terms of field  MLT – more like this 11

12. SolrJ  Library to connect and interact with Solr 12

13. Thank You! 13

Editor's Notes

<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler"> <lst name="defaults"> <str name="config">data.import.xml</str> </lst> </requestHandler> --------------------------------------------- <dataConfig> <dataSource type="JdbcDataSource" driver="com.microsoft.sqlserver.jdbc.SQLServerDriver" url="jdbc:sqlserver://TW-TSDev01;databaseName=SPTContentV2;loginTimeout=1800;queryTimeout=60000;" user="kb_reader" password="p@ssw0rd@0" batchSize="5" responseBuffering="adaptive" /> <document name="KB"> <entity name="KBEntity" dataSource="JdbcDataSource" transformer="HTMLStripTransformer" query="spGetKBSolution"> <field column="Id" name="Id"/> <field column="PublicContent" name="PublicContent" stripHTML="true"/> </entity> </document> </dataConfig> -----------------------------------------------------------------
query="SELECT [SolutionId],[Locale],[Title],[Problems],[PublicContent],[Modified] FROM [uvw_KBSolution]“ deltaImportQuery="SELECT [SolutionId],[Locale],[Title],[Problems],[PublicContent],[Modified] FROM [uvw_KBSolution] where SolutionId='${dih.delta.SolutionId}'“ deltaQuery="select SolutionId from [uvw_KBSolution] where Modified > '${dih.last_index_time}'"
#Tue Jul 21 12:10:50 CEST 2010 metadataObject.last_index_time=2010-09-20 11\:12\:47 last_index_time=2010-09-20 11\:12\:47 # to sync or not to sync # 1 - active; anything else - inactive syncEnabled=1 # which cores to schedule # in a multi-core environment you can decide which cores you want syncronized # leave empty or comment it out if using single-core deployment syncCores=esupport,lithium # solr server name or IP address # [defaults to localhost if empty] server=localhost # solr server port # [defaults to 80 if empty] port=8080 # application name/context # [defaults to current ServletContextListener's context (app) name] webapp=solr # URL params [mandatory] # remainder of URL params=/select?qt=/dataimport&command=delta-import&clean=false&commit=true # schedule interval # number of minutes between two runs # [defaults to 30 if empty] interval=10
q=osce&start=0&rows=2&wt=xml&indent=true &defType=edismax &lowercaseOperators=true &hl=true&hl.fl=Title%2CSegment%2CProblems%2CPublicContent%2CSolutionId &hl.simple.pre=%3Cem%3E&hl.simple.post=%3C%2Fem%3E&hl.usePhraseHighlighter=true&hl.highlightMultiTerm=true &facet=true&facet.query=osce&facet.field=Segment,OperatingSystem &spellcheck=true&spellcheck.build=true&spellcheck.reload=true &spellcheck.q=oscc&spellcheck.onlyMorePopular=true&spellcheck.accuracy=0.7

Introduction to Lucene and Solr - 2

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

More from YI-CHING WU

More from YI-CHING WU (7)

Recently uploaded

Recently uploaded (20)

Introduction to Lucene and Solr - 2

Editor's Notes