This document provides the administration guide for IDOL Server version 7.3, describing how to install, configure, and operate the IDOL Server software. It covers topics such as installing IDOL Server, configuring content storage and indexing, processing data, using fields and language support, working with agents, categorization, clustering, and other IDOL Server operations. The document is organized into parts that cover getting started, storing content, IDOL server operations, and includes chapters on specific functions and processes within the IDOL Server system.
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Idol Server 7.3 Admin Rev7
1. IDOL Server
7.3
Administration Guide
Revision 7
17 September 2008
2. Copyright Notice
Notice
This document is a proprietary product of Autonomy and is protected by copyright laws and international treaty. Information
in this manual is subject to change without notice and does not represent a commitment on the part of Autonomy. While
reasonable efforts have been made to ensure the accuracy of the information contained herein, Autonomy assumes no
liability for errors or omissions. No liability is assumed for direct, incidental, or consequential damages resulting from the use
of the information contained in this document.
The copyrighted software that accompanies this document is licensed to the End User for use only in strict accordance with
the End User License Agreement, which the Licensee should read carefully before commencing use of the software. No part
of this publication may be reproduced, transmitted, stored in a retrieval system, nor translated into any human or computer
language, in any form or by any means, electronic, mechanical, magnetic, optical, chemical, manual or otherwise, without
the prior written permission of the copyright owner.
This document may use fictitious names for purposes of demonstration; references to actual persons, companies, or
organizations is strictly coincidental.
Trademarks and Copyrights
Copyright 2008 Autonomy Corporation plc and all its affiliates. All rights reserved. Autonomy AnswerTM, Autonomy Desktop
SuiteTM, DAHTM, DIHTM, DiSHTM, DLCMTM, DLHTM, DOHTM, Enterprise AWETM, IASTM, IDOLTM, IDOL serverTM, IDOLmeTM,
IDOL DesktopTM, Portal-in-a-BoxTM, RetinaTM, ACI APITM, Active KnowledgeTM, Active SDKTM, ActiveServerTM, AutoIndexerTM,
Autonomy LiteTM, Autonomy Offline CD PublisherTM, Autonomy POD for Siebel eBusiness ApplicationsTM, Document
Management ServerTM, Import ModuleTM, SpeechPluginTM, VoiceSuiteTM, Autonomy Fetch for Siebel eBusiness
ApplicationsTM, Documentum FetchTM, eRoom FetchTM, Exchange FetchTM, FatWire FetchTM, FileNet FetchTM, FileNet P8
FetchTM, File System FetchTM, FTP FetchTM, HTTP FetchTM, HummingbirdDM FetchTM, IBM Content Manager FetchTM, IBM
Workplace FetchTM, iManage FetchTM, KVS FetchTM, Moreover FetchTM, NNTP FetchTM, Notes FetchTM, ODBC FetchTM, Omni
Fetch SDKTM, OpenText FetchTM, Oracle FetchTM, PCDocs FetchTM, POP3 FetchTM, SAP FetchTM, Schlumberger FetchTM,
SharePoint FetchTM, Stellent FetchTM, VirageTM, VidoeLoggerTM, AudioLoggerTM, SmartencodeTM, Virage ControlCenterTM, VS
ArchiveTM, VS Broadcast MonitoringTM, VerityTM, CardiffTM, the Verity logo, the LiquidPDF logo, KeyViewTM, UltraseekTM,
Knowledge OrganizerTM, TOPICTM, Verity Portal OneTM, Verity ProfilerTM, LiquidOfficeTM, LiquidPDFTM, Connect AgentTM,
HTML+FormsTM, MediClaimTM, PDF+FormsTM, TeleFormTM, Tri-CRTM, RecoFlexTM, AutoMerge PublisherTM, TrueAddressTM,
and VersiFormTM are trademarks of the Autonomy group of companies.
Microsoft is a registered trademark, and MS-DOS, Windows, Windows 95, Windows NT, SharePoint, and other Microsoft
products referenced herein are trademarks of Microsoft Corporation.
UNIX is a registered trademark of The Open Group.
AvantGo is a trademark of AvantGo, Inc.
Epicentric Foundation Server is a trademark of Epicentric, Inc.
Documentum and eRoom are trademarks of Documentum, a division of EMC Corp.
FileNet is a trademark of FileNet Corporation.
Lotus Notes is a trademark of Lotus Development Corporation.
mySAP Enterprise Portal is a trademark of SAP AG.
Oracle is a trademark of Oracle Corporation.
Stellent is a trademark of Stellent, Inc.
All other trademarks are the property of their respective owners.
Notice to Government End Users
If this product is acquired under the terms of a DoD contract: Use, duplication, or disclosure by the Government is subject
to restrictions as set forth in subparagraph (c)(1)(ii) of 252.227-7013. Civilian agency contract: Use, reproduction or
disclosure is subject to 52.227-19 (a) through (d) and restrictions set forth in the accompanying end user agreement.
Unpublished-rights reserved under the copyright laws of the United States. Autonomy, Inc., One Market Plaza, Spear Tower,
Suite 1900, San Francisco, CA. 94105, US.
17.09.08
3. Contents
Figures, Tables, and Listings .................................................................................................. 19
Preface .............................................................................................................................. 21
About This Book ...................................................................................................................... 21
Version ............................................................................................................................. 21
Organization of This Book ................................................................................................ 22
Stylistic Conventions ........................................................................................................ 22
Related Documentation........................................................................................................... 24
Support and Contact Information ............................................................................................ 25
Downloading the Latest Documentation........................................................................... 25
Autonomy Technical Support ........................................................................................... 26
Contact Autonomy............................................................................................................ 26
PART 1 GET STARTED
1 Introduction to IDOL Server ........................................................................................ 29
Autonomy Infrastructure .......................................................................................................... 30
Administration................................................................................................................... 32
IDOL Server ..................................................................................................................... 32
Connectors ....................................................................................................................... 33
Interfaces.......................................................................................................................... 33
Distributed systems .......................................................................................................... 33
PODS ............................................................................................................................... 33
IDOL Server Operations.......................................................................................................... 34
Agents .............................................................................................................................. 34
Alerting ............................................................................................................................. 35
Automatic Query Guidance .............................................................................................. 35
Categorization .................................................................................................................. 35
Channels .......................................................................................................................... 36
Clustering ......................................................................................................................... 36
3
4. Contents
Collaboration..................................................................................................................... 36
Dynamic Clustering........................................................................................................... 37
Dynamic Thesaurus.......................................................................................................... 37
Eduction............................................................................................................................ 37
Expertise........................................................................................................................... 37
Hyperlinks ......................................................................................................................... 38
Mailing .............................................................................................................................. 38
Profiling............................................................................................................................. 38
Search and Retrieval ........................................................................................................ 39
Spelling Correction ........................................................................................................... 41
Summarization.................................................................................................................. 41
Taxonomy Generation ...................................................................................................... 42
IDOL System Architecture ....................................................................................................... 42
General ............................................................................................................................. 42
Index and Query ............................................................................................................... 43
Security............................................................................................................................. 44
2 Install IDOL ....................................................................................................................... 47
System Requirements ............................................................................................................. 47
Basic Requirements.......................................................................................................... 47
Supported Platforms ......................................................................................................... 48
Minimum Hardware Specifications ................................................................................... 48
Recommended Hardware Specifications.......................................................................... 48
Install IDOL.............................................................................................................................. 49
Install IDOL Server Standalone ........................................................................................ 49
Installed Directory Structure .................................................................................................... 63
Licenses .................................................................................................................................. 67
Display License Information.............................................................................................. 68
Revoke a Client License ................................................................................................... 69
Forcibly Revoke Licenses from Inaccessible Clients........................................................ 69
Troubleshoot License Errors............................................................................................. 70
3 Run IDOL Server ............................................................................................................. 73
Start and Stop IDOL Server..................................................................................................... 73
Start IDOL Server ............................................................................................................. 73
Stop IDOL Server ............................................................................................................. 74
4 IDOL Server 7.3 Administration Guide Revision 7
5. Contents
Send Action Commands to IDOL Server ................................................................................ 75
Verify IDOL Server Runs Correctly ......................................................................................... 76
GetRequestLog ................................................................................................................ 76
GetLicenseInfo ................................................................................................................. 76
GetStatus ......................................................................................................................... 77
Display Online Help................................................................................................................. 77
PART 2 STORE CONTENT IN IDOL SERVER
4 Configure Content Storage ......................................................................................... 81
Configure Through the IDOL Dashboard ................................................................................ 81
Edit the Configuration File ....................................................................................................... 82
Use a Unified Configuration .................................................................................................... 82
Stored Content ........................................................................................................................ 82
Disable Content Storage .................................................................................................. 83
Store IDOL Server Data Files on Multiple Disks .............................................................. 83
Allocate Files to IDOL Server Databases ......................................................................... 84
Set Up Field Index Process..................................................................................................... 86
Index XML Attributes ........................................................................................................ 87
Configure IDOL Server for Language and to Encode ............................................................. 89
Optimize Index Process .......................................................................................................... 90
Index Process................................................................................................................... 90
Delayed Synchronization.................................................................................................. 90
5 Process Data Before Indexing.................................................................................... 93
Pre-Indexing Tasks ................................................................................................................. 94
Set Up Pre-Indexing Tasks ..................................................................................................... 95
Autogenerate Document Titles................................................................................................ 96
Autogenerate Titles for a Different IDOL Server ..................................................................... 97
Categorize and Add Data to a SQL Database ........................................................................ 98
Simple Route of Documents According to Type ..................................................................... 99
Advanced Route of Documents by Type............................................................................... 101
Use Lua Script Index Tasks .................................................................................................. 103
Requirements ................................................................................................................. 103
Configure a Lua Indexing Task ...................................................................................... 103
Write a Lua Index Task .................................................................................................. 104
IDOL Server 7.3 Administration Guide Revision 7 5
6. Contents
Change the Value of a Field ........................................................................................... 105
Add a Field...................................................................................................................... 105
Sections .......................................................................................................................... 105
Example Script................................................................................................................ 106
Process Documents with Repeated Fields ..................................................................... 107
Use Add2Replace.................................................................................................................. 108
Supported Configuration Parameters ............................................................................. 108
Add2Replace Example ................................................................................................... 108
6 Index Data ........................................................................................................................ 111
Index Overview...................................................................................................................... 112
DREADD: Index IDX and XML Files Directly......................................................................... 113
DREADD Parameters ..................................................................................................... 114
Basic Examples .............................................................................................................. 117
Specify Field Names....................................................................................................... 117
DREADDDATA: Index Data Over a Socket........................................................................... 120
DREADDDATA Parameters ........................................................................................... 120
Examples ........................................................................................................................ 121
Index StopWords ................................................................................................................... 122
Index Non-Alphanumeric Characters .................................................................................... 123
Term Separators............................................................................................................. 123
Index Non-Alphanumeric Characters for Retrieval ......................................................... 124
Hyphenated Terms ......................................................................................................... 125
Prevent Duplicate Documents ............................................................................................... 126
Deduplication Options—KillDuplicates and IndexMode.................................................. 127
Enable Deduplication for all Index Jobs.......................................................................... 128
Enable Deduplication for Individual Index Jobs .............................................................. 130
Enable Deduplication for Connector Index Jobs............................................................. 131
Deduplication Constraints............................................................................................... 131
Add Metadata to Documents ................................................................................................. 132
Check Index Status ............................................................................................................... 132
Tag Documents into Clusters ................................................................................................ 135
6 IDOL Server 7.3 Administration Guide Revision 7
7. Contents
7 Fields ................................................................................................................................ 139
About Fields .......................................................................................................................... 140
Process Fields....................................................................................................................... 143
Index Fields ........................................................................................................................... 147
Configure the Number Index Process ................................................................................... 149
NumericDateType Fields....................................................................................................... 150
NumericType Fields .............................................................................................................. 152
FieldCheckType Fields.......................................................................................................... 153
ReferenceType Fields ........................................................................................................... 156
Set up ReferenceType fields .......................................................................................... 156
Use KillDuplicates and Combine on ReferenceType fields ............................................ 158
Highlight Fields...................................................................................................................... 160
Agentboolean Fields ............................................................................................................. 161
Store Agents in Agentboolean Fields ............................................................................. 161
Match Documents Against Agentboolean Categories .................................................... 162
Metafields .............................................................................................................................. 165
Change Field Values ............................................................................................................. 166
8 Language Support ....................................................................................................... 167
IDOL Language-Support Concepts ....................................................................................... 167
Run IDOL Server in Multiple Languages............................................................................... 170
Check Those Languages that Are Enabled.................................................................... 171
Define Language Types ................................................................................................. 172
Associate Language Types with Documents ................................................................. 174
Add Language-Type Fields to Documents ..................................................................... 177
Define a Default Language Type.................................................................................... 177
Define a General Language ........................................................................................... 178
Enable Automatic Language Detection .......................................................................... 178
Specify the Language Type of a Query.......................................................................... 180
Convert Results to a Specific Encoding ......................................................................... 180
Return Documents in Multiple Languages ..................................................................... 181
Return Documents in a Specific Language .................................................................... 182
Create a Custom Stem File for a Language................................................................... 183
Decompose Compound Words ...................................................................................... 184
Enable Transliteration for a Language ........................................................................... 184
IDOL Server 7.3 Administration Guide Revision 7 7
8. Contents
PART 3 IDOL SERVER OPERATIONS
9 Agents .............................................................................................................................. 189
About Agents ......................................................................................................................... 189
Manipulate Agents................................................................................................................. 190
Create an Agent.............................................................................................................. 190
Edit an Agent .................................................................................................................. 190
Retrain an Agent............................................................................................................. 191
Copy an Agent ................................................................................................................ 191
View an Agent’s Details .................................................................................................. 191
Delete an Agent .............................................................................................................. 192
Query with Agents ................................................................................................................. 192
Alert with Agents.................................................................................................................... 193
Alert Users to New Content ............................................................................................ 193
Create Templates for Alert E-mails................................................................................. 195
Collaboration and Expertise with Agents............................................................................... 197
Collaboration................................................................................................................... 197
Expertise......................................................................................................................... 197
10 Categorization ............................................................................................................... 199
Introduction to Categorization................................................................................................ 199
Create a Hierarchical Category Structure.............................................................................. 200
Create Categories from Scratch ..................................................................................... 201
Create Categories from Clusters .................................................................................... 202
Create Categories from Legacy Topic Sets.................................................................... 202
Create Categories by Copying Categories ..................................................................... 203
Create Categories when you Generate a Taxonomy ..................................................... 203
Create Categories from XML .......................................................................................... 203
Train Categories ............................................................................................................. 204
Retrain Categories.......................................................................................................... 204
Move Categories............................................................................................................. 205
View and Administer Categories ........................................................................................... 205
View Category Details .................................................................................................... 206
View Category Hierarchy Details .................................................................................... 206
View Category Terms and Weights ................................................................................ 206
View Category Training .................................................................................................. 206
8 IDOL Server 7.3 Administration Guide Revision 7
9. Contents
Change Category Fields................................................................................................. 207
Reset Category Fields.................................................................................................... 207
Change Category Term Weights .................................................................................... 207
Remove Category Term Weights ................................................................................... 208
Replace Categories........................................................................................................ 208
Activate or Deactivate Categories .................................................................................. 208
Build Categories ............................................................................................................. 209
Delete Categories........................................................................................................... 209
Delete Category Training ............................................................................................... 209
Export Categories to XML .............................................................................................. 210
Synchronize IDOL Server with Stored Categories ......................................................... 210
Categorize Data .................................................................................................................... 210
Suggest Categories............................................................................................................... 211
Suggest Categories for Documents ............................................................................... 211
Suggest Categories for Text........................................................................................... 211
Suggest Categories for Categories ................................................................................ 212
Match Categories .................................................................................................................. 212
Create Taxonomies ............................................................................................................... 212
Generate Taxonomies Automatically ............................................................................. 213
Create Named Taxonomies ........................................................................................... 214
Example of Categorization Use............................................................................................. 215
11 Binary Categories......................................................................................................... 217
About Binary Categories ....................................................................................................... 217
Create and Administer Binary Categories ............................................................................. 218
Create a Binary Category ............................................................................................... 218
Train a Binary Category ................................................................................................. 218
Delete Training From a Binary Category ........................................................................ 219
Change a Binary Category’s Details .............................................................................. 219
View a Binary Category’s Details ................................................................................... 219
List a System’s Binary Categories.................................................................................. 220
Delete a Binary Category ............................................................................................... 220
Query with Binary Categories ............................................................................................... 220
Example of Binary Category Use .......................................................................................... 221
IDOL Server 7.3 Administration Guide Revision 7 9
10. Contents
12 Cluster Process ............................................................................................................. 223
Generate Snapshots.............................................................................................................. 224
Generate Spectrograph Data ................................................................................................ 226
Generate WhatsNew and WhatsHot Information .................................................................. 227
WhatsNew information.................................................................................................... 228
WhatsHot information ..................................................................................................... 228
Generate a Cluster Map After You Cluster ..................................................................... 229
Configure Clusters................................................................................................................. 229
Change the Number and Size of Clusters ...................................................................... 230
Set Up Schedules.................................................................................................................. 234
13 Profiles ............................................................................................................................. 237
About Profiles ........................................................................................................................ 237
Profile a User......................................................................................................................... 238
Create an Interest Profile for a User ............................................................................... 238
Create an Expertise Profile for a User ............................................................................ 238
Manipulate Profiles ................................................................................................................ 239
Edit a Profile ................................................................................................................... 239
Query with a Profile ........................................................................................................ 240
View a Profile’s Details ................................................................................................... 240
Delete a Profile ............................................................................................................... 240
Collaboration and Expertise with Profiles .............................................................................. 241
Collaboration................................................................................................................... 241
Expertise......................................................................................................................... 241
PART 4 RESULTS
14 Search and Retrieve..................................................................................................... 245
Actions................................................................................................................................... 246
Conceptual Matches.............................................................................................................. 247
Types of Matches ........................................................................................................... 247
Example Queries ............................................................................................................ 248
Keyword Search .................................................................................................................... 250
Keyword Occurrence Search.......................................................................................... 250
Exact Keyword Search ................................................................................................... 250
10 IDOL Server 7.3 Administration Guide Revision 7
11. Contents
Case-Sensitive Exact Keyword Search.......................................................................... 251
Keyword Search Examples ............................................................................................ 251
Phrase Search ...................................................................................................................... 256
Phrase Occurrence Search ............................................................................................ 256
Default Phrase Search ................................................................................................... 257
Exact Phrase Search...................................................................................................... 257
Case-Sensitive Exact Phrase Search ............................................................................ 258
Phrase Search Examples ............................................................................................... 258
Boolean and Proximity Search .............................................................................................. 262
Boolean Search Operators............................................................................................. 262
Proximity Search Operators ........................................................................................... 263
WHEN Search Operator................................................................................................. 265
Precedence of Search Operators ................................................................................... 269
Simple Field Restricted Search ............................................................................................. 269
Field Text Search .................................................................................................................. 270
Field Text Queries Guidelines ........................................................................................ 271
Field Specifiers for Common Restrictions ...................................................................... 272
Field Specifiers for Advanced Restrictions ..................................................................... 282
Field Specifiers to Bias Result Scores ........................................................................... 305
Fuzzy Search ........................................................................................................................ 306
Fuzzy Query Syntax ....................................................................................................... 306
Adjust the Tolerance Level of a Fuzzy Search............................................................... 307
Parametric Search ................................................................................................................ 307
Configure IDOL Server for Parametric Fields................................................................. 308
Execute a Parametric Search......................................................................................... 309
Proper Names Search........................................................................................................... 311
Enable Proper Names Searches .................................................................................... 311
Example Proper Name Searches ................................................................................... 314
Soundex Keyword Search..................................................................................................... 315
Enable Soundex Keyword Searches.............................................................................. 315
Execute Soundex Keyword Searches ............................................................................ 316
Synonym Search ................................................................................................................... 316
Enable Synonym Searches ............................................................................................ 316
Set Up an Additional Synonym IDOL Server.................................................................. 320
Verity Query Language Search ............................................................................................. 322
Convert Other Query Types to VQL ............................................................................... 322
IDOL Server 7.3 Administration Guide Revision 7 11
12. Contents
Combine Different Search Types .......................................................................................... 325
Synonym and Boolean Searches ................................................................................... 325
Synonym Search and Field Restrictions......................................................................... 325
Soundex and Proper Names Searches .......................................................................... 325
Soundex and Boolean Searches .................................................................................... 326
Soundex and Proximity Searches................................................................................... 326
Soundex Search and Field Restrictions.......................................................................... 326
Exact Phrase and Boolean Searches ............................................................................. 326
Exact Phrase and Proximity Searches ........................................................................... 327
Exact Phrase Search and Field Restrictions .................................................................. 327
Boolean and Proximity Searches.................................................................................... 328
Boolean Search and Field Restrictions........................................................................... 328
Proximity Searches and Field Restrictions ..................................................................... 328
Wildcards in Queries ............................................................................................................. 329
Wildcards in Query Text ................................................................................................. 329
Wildcards in Field Text Queries...................................................................................... 331
Wildcard Searches in Japanese, Chinese, Korean and Thai ......................................... 333
Query for Non-Alphanumeric Characters .............................................................................. 333
Text................................................................................................................................. 334
FieldText ......................................................................................................................... 334
Optimize Retrieval of Tagged Documents............................................................................. 336
Query Syntaxes .............................................................................................................. 336
15 Customize Results ....................................................................................................... 339
Change the Results Display .................................................................................................. 340
Set the Number of Results to Display............................................................................. 340
Change Result Sorting (Display Order) .......................................................................... 340
Batch (Page) Results...................................................................................................... 345
Change the Field Display ...................................................................................................... 346
Returned Fields .............................................................................................................. 346
Display Additional Metafields.......................................................................................... 346
Display Document Fields ................................................................................................ 346
Use XSL Templates to Change Output Format..................................................................... 349
Enable the XSL Templates ............................................................................................. 350
Apply XSL Templates to Actions .................................................................................... 351
12 IDOL Server 7.3 Administration Guide Revision 7
13. Contents
Generate Summaries ............................................................................................................ 352
Types of Summaries ...................................................................................................... 352
Return Summaries with Query Results .......................................................................... 352
Summarize Text or Documents ...................................................................................... 354
Cluster Results ...................................................................................................................... 354
Generate Hyperlinks ............................................................................................................. 356
About Hyperlinks ............................................................................................................ 356
Implement Hyperlinks ..................................................................................................... 357
Provide Spell Correction ....................................................................................................... 357
How Spell Correction Works .......................................................................................... 357
Spell Correction File ....................................................................................................... 358
Automatic Query Guidance ................................................................................................... 359
About Automatic Query Guidance .................................................................................. 360
Enable Automatic Query Guidance ................................................................................ 361
About the QuerySummary Parameter ............................................................................ 361
Generate Query Summaries (Dynamic Thesaurus) .............................................................. 363
About Query Summaries ................................................................................................ 363
Generating Query Summaries........................................................................................ 364
Generate Dynamic Clusters .................................................................................................. 366
Display Cluster Information ............................................................................................ 368
Display the Number of Documents a Dynamic Cluster Contains ................................... 369
Create a Cluster Map ..................................................................................................... 371
16 Manipulate Result Relevance ................................................................................... 373
Boost Relevance ................................................................................................................... 373
Use a Field Process to Boost Relevance .............................................................................. 374
Use the BIAS Field Specifier to Boost Relevance................................................................. 377
Use Multipliers to Boost Relevance ...................................................................................... 379
Use the AutnRankType Field to Boost Relevance ................................................................ 380
17 Manipulate the Results Set........................................................................................ 383
Combine Parameter .............................................................................................................. 383
Simple ............................................................................................................................ 384
FieldCheck ..................................................................................................................... 384
ReferenceTypeFields ..................................................................................................... 385
FieldCheck Parameter .......................................................................................................... 386
IDOL Server 7.3 Administration Guide Revision 7 13
14. Contents
Predict Parameter.................................................................................................................. 387
Store and Retrieve the Result State ...................................................................................... 387
Store the Result State..................................................................................................... 388
Query with the State Token ............................................................................................ 388
Use a State Token with Index Commands ..................................................................... 389
PART 5 ADMINISTRATION AND MAINTENANCE
18 Set up Security .............................................................................................................. 393
Set up Security on Documents .............................................................................................. 393
Set up a SSL Connection ...................................................................................................... 397
Set Up SSL for Shared Communications ....................................................................... 399
Log SSL Settings ............................................................................................................ 400
Set up SSL for Mailer...................................................................................................... 400
19 Add Users to IDOL Server.......................................................................................... 401
When to Create Users ........................................................................................................... 401
Create IDOL Users ................................................................................................................ 402
Flat Structure .................................................................................................................. 402
Hierarchical Structure ..................................................................................................... 403
Integrate with a Third-Party User Structure ........................................................................... 403
20 Mail .................................................................................................................................... 405
Automatically E-mail Agent and Channel Results ................................................................. 405
Send Custom E-mails............................................................................................................ 408
Send E-mails in Batches ....................................................................................................... 409
Mailer Templates ................................................................................................................... 410
Edit Templates................................................................................................................ 411
21 Administer IDOL Server .............................................................................................. 415
Execute Configuration Changes............................................................................................ 416
Delete and Restore Documents ............................................................................................ 416
Delete Documents by Reference.................................................................................... 416
Delete Documents and Ranges of Documents............................................................... 418
Restore Deleted Documents........................................................................................... 419
14 IDOL Server 7.3 Administration Guide Revision 7
15. Contents
Create and Delete Databases ............................................................................................... 420
Create a New Database ................................................................................................. 420
Delete a Database and All its Documents...................................................................... 422
Delete All Documents from a Database ......................................................................... 423
Expire Documents ................................................................................................................. 423
Set Up a Field Process................................................................................................... 424
Expire Immediately......................................................................................................... 425
Expire at Regular Intervals ............................................................................................. 426
Change Document Metadata ................................................................................................ 427
Change Document Field Values ........................................................................................... 428
Compact the Data Index ....................................................................................................... 431
Compact the Data Index Immediately ............................................................................ 432
Compact the Data Index at Regular Intervals ................................................................ 432
Initialize the Data Index ......................................................................................................... 433
22 Back up IDOL Server ................................................................................................... 435
Back Up Content ................................................................................................................... 435
Back Up an IDOL Server’s Entire Data Index ................................................................ 436
Export IDX Documents from IDOL Server...................................................................... 439
Export XML Documents from IDOL Server .................................................................... 441
Restore Content .................................................................................................................... 443
Export Users, Roles, Agents, and Profiles ............................................................................ 443
Import Users, Roles, Agents, and Profiles ............................................................................ 444
Back up Categories, Taxonomies, and Cluster Jobs ............................................................ 445
Restore Categories, Taxonomies, and Cluster Jobs............................................................. 446
23 Troubleshoot IDOL Server ......................................................................................... 447
IDOL Server Log Files........................................................................................................... 447
Set Up Log Streams .............................................................................................................. 449
IDOL Statistics Server ........................................................................................................... 451
IDOL Server 7.3 Administration Guide Revision 7 15
18. Contents
D Manually Create IDX Files .......................................................................................... 535
IDX Format ............................................................................................................................ 535
Section a Document .............................................................................................................. 538
E Category XML Format ................................................................................................. 541
Introduction............................................................................................................................ 541
XML Format........................................................................................................................... 542
Example................................................................................................................................. 551
F Record Statistics with Statistics Server ................................................................ 553
About Statistics Server .......................................................................................................... 553
Configuration ......................................................................................................................... 554
Create XML events ......................................................................................................... 555
Configure Statistics Server Information .......................................................................... 556
Define Statistical Criteria ................................................................................................ 556
Record and View Statistics.................................................................................................... 557
Record Statistics............................................................................................................. 557
View Statistical Results................................................................................................... 558
Record Statistics from Multiple IDOL Servers ....................................................................... 559
Preserve Data during Service Interruptions........................................................................... 560
Sample Files.......................................................................................................................... 561
Sample Configuration File .............................................................................................. 561
Sample XML Event Script............................................................................................... 563
Configuration Parameter Reference...................................................................................... 570
Statistics Server Parameters .......................................................................................... 570
Statistical Criteria Parameters ........................................................................................ 575
Action and Action Parameter Reference ............................................................................... 578
Event .............................................................................................................................. 579
GetStatus ....................................................................................................................... 580
StatResult ...................................................................................................................... 580
G Error Codes and Messages ....................................................................................... 583
Error Codes ........................................................................................................................... 583
Error Messages ..................................................................................................................... 584
VQL Conversion Error Messages ................................................................................... 584
General Errors ................................................................................................................ 585
18 IDOL Server 7.3 Administration Guide Revision 7
19. Contents
Proximity Errors.............................................................................................................. 585
Boolean Operator Errors ................................................................................................ 587
Field Restriction Errors ................................................................................................... 588
Word and Phrase Errors................................................................................................. 589
Expression Errors........................................................................................................... 590
Glossary .......................................................................................................................... 591
Index ................................................................................................................................. 597
IDOL Server 7.3 Administration Guide Revision 7 19
20. Contents
20 IDOL Server 7.3 Administration Guide Revision 7
21. Figures, Tables, and Listings
Figure 1-1 Types of business information ................................................................... 30
Figure 1-2 ACI client API ............................................................................................ 43
Figure 1-3 Index and Query IDOL server.................................................................... 44
Table 2-1 IDOL standalone installation structure....................................................... 64
Table 2-2 License-related error messages ................................................................ 70
Table 6-1 IndexerGetStatus results ......................................................................... 133
Table 14-1 Boolean search operators ....................................................................... 262
Table 14-2 Proximity search operators ...................................................................... 264
Table B-1 Supported Encodings .............................................................................. 517
Table B-2 Recommended TermSize value Per Language....................................... 518
Table D-1 IDOL Server fields in IDX files ................................................................. 535
19
22. Figures, Tables, and Listings
20 IDOL Server 7.3 Administration Guide Revision 7
23. Preface
This guide is for IDOL server users and administrators. It is intended for readers who are
responsible for setting up, using, and maintaining an IDOL server installation and who are
familiar with enterprise software administration.
This preface contains the these sections:
About This Book
Related Documentation
Support and Contact Information
About This Book
This section briefly describes the organization of this book and the stylistic conventions it
uses.
Version
The information in this book is current as of IDOL server 7.3. The content was last modified 16
September 2008. Corrections or updates to this information may be available through the
Autonomy Support site; see “Support and Contact Information” on page 25.
21
24. Preface
About This Book
Organization of This Book
This book includes these parts:
Part 1, “Get Started” gives an overview of IDOL server and describes how to install and run
the server.
Part 2, “Store Content in IDOL Server” explains the concept of indexing and describes how
to index document content and metadata into IDOL server.
Part 3, “IDOL Server Operations” shows how you can make best use of the many
information-retrieval, analysis, classification, and management capabilities of IDOL server.
Part 4, “Results” describes the processes of querying IDL server, retrieving results, and
displaying those results to users.
Part 5, “Administration and Maintenance” explains how to administer your IDOL server
installation and how to perform routine maintenance.
Appendixes list important information such as error codes, supported language settings,
indexing and category file formats, and commands.
A glossary of terms is also provided.
Stylistic Conventions
These stylistic conventions are used in this book.
Convention Usage
Plain Narrative text.
Bold User-interface elements in narrative text:
Click Cancel to halt the operation.
Italics Book titles and new terms:
For more information, refer to the IDOL Server
Administration Guide.
An action command is a request, such as a query or
instruction, sent to IDOL Server.
22 IDOL Server 7.3 Administration Guide Revision 7
25. Preface
About This Book
Convention Usage
Monospace File names, paths, and code:
The name.ext file is installed in:
C:AutonomyData
Monospace italic Replaceable strings in file paths and code:
user username
Monospace bold Data types and required user input:
SrvConnect A connection handle.
In the User Interface text box, type user1.
These command-line syntax conventions are used in this book.
Convention Usage
[ optional ] Brackets describe optional syntax, as in [ -create ] to specify a
non-required option.
| Bars indicate “either | or” choices, as in
[ option1 ] | [ option2 ]
In this example, you must choose between option1 and option2.
{ required } Braces describe required syntax in which you have a choice and that at
least one choice is required, as in
{ [ option1 ] [ option2 ] }
In this example, you must choose option1, option2, or both options.
required Absence of braces or brackets indicates required syntax in which there is no
choice; you must enter the required syntax element.
metavariable Italics specify items to be replaced by actual values, as in
-merge filename1
<metavariable> (In some documents, angle brackets are used to denote these items.)
... Ellipses indicate repetition of the same pattern, as in
-merge filename1, filename2 [, filename3 ... ]
where the ellipses specify , filename4, and so on.
IDOL Server 7.3 Administration Guide Revision 7 23
26. Preface
Related Documentation
Use of punctuation—such as single and double quotes, commas, periods—indicates actual
syntax; it is not part of the syntax definition.
Related Documentation
These documents provide more details on Autonomy products that are closely related to IDOL
server.
IDOL Administration User Guide
Distributed Action Handler (DAH) Administration Guide
Distributed Indexing Handler (DIH) Administration Guide
Distributed Service Handler (DiSH) Administration Guide
Intellectual Asset Protection System (IAS) Administration Guide
Eduction User Guide
Business Console Taxonomy Management User Guide
Business Console Content Management User Guide
File System Connector Administration Guide, HTTP Connector Administration Guide, and
other connector guides.
24 IDOL Server 7.3 Administration Guide Revision 7
27. Preface
Support and Contact Information
Support and Contact Information
Read this section if you want to contact Autonomy, request technical support, or obtain
product documentation.
Downloading the Latest Documentation
You can retrieve the latest available product documentation from Autonomy’s Knowledge
Base on the Customer Support site.
To download the latest document revisions
1. Enter this URL in your Web browser's address field:
https://customers.autonomy.com
2. Access to the contents of the Customer Support site requires a user name and password.
To obtain a user name and password, follow the sign-up instructions on the home page.
3. Click Log In and enter your email address and password. Click Sign in.
4. Click Knowledge Base.
5. Enter query text or select a category to filter your search for documentation.
The most recently added documents are displayed at the top of the results list.
The version number associated with a document (for example, version 7.4) is the product
version that the document describes. If a document has a revision number (for example,
Revision 5), the document has been revised since it was first released with the specified
product version. The Knowledge Base contains the latest available revision of any document.
IDOL Server 7.3 Administration Guide Revision 7 25
28. Preface
Support and Contact Information
Autonomy Technical Support
Autonomy Technical Support exists to provide you with prompt and accurate resolutions to
difficulties relating to using Autonomy software products. You can contact Technical Support
using any of the following methods.
Call or email the support group at the location that is nearest to you:
Europe and Worldwide North and South America
Telephone: Telephone:
00 800 4837 4890 (UK, Germany, Spain, 1 877 333 7744
Netherlands, France) 1 (403) 294 1107 (Canada direct)
+44 (0) 800 0 282 858 (UK only) Hours: 07:00 to 18:00 MST (GMT-7)
Hours: 09:00 to 17:00 (GMT+1) Email: support@us.autonomy.com
Email: tsp-europe@autonomy.com
Access the Customer Support site, at
https://customers.autonomy.com
Access to the contents of the Customer Support site requires a user name and password. To
obtain a user name and password, follow the sign-up instructions on the home page.
Contact Autonomy
Contact the location that is nearest to you for general information about Autonomy:
Europe and Worldwide North and South America
Autonomy Corporation plc Autonomy, Inc.
Cambridge Business Park One Market Plaza, Spear Tower, Suite 1900
Cowley Road, Cambridge, CB4 0WZ, UK San Francisco, CA. 94105, US
Telephone: +44 (0) 1223 448 000 Telephone: 1 (415) 243 9955
Fax: +44 (0) 1223 448 001 Fax: 1 (415) 243 9984
General information email: General information email:
autonomy@autonomy.com autonomy@autonomy.com
26 IDOL Server 7.3 Administration Guide Revision 7
29. 1 Get Started
Chapter 1, “Introduction to IDOL Server”
Chapter 2, “Install IDOL”
Chapter 3, “Run IDOL Server”
27
31. 1 Introduction to IDOL Server
Using Autonomy Connectors, Autonomy's Intelligent Data Operating Layer (IDOL) server
integrates unstructured, semi-structured and structured information from multiple repositories
through an understanding of the content, delivering a real time environment in which
operations across applications and content are automated, removing all the manual
processes involved in getting the right information to the right people, at the right time.
Autonomy Infrastructure
IDOL Server Operations
IDOL System Architecture
29
32. 1 Introduction to IDOL Server
Autonomy Infrastructure
Autonomy Infrastructure
Information that you need to conduct business successfully includes these types.
Figure 1-1 Types of business information
In the past, companies could only make use of 20% of the information that was relevant to
them. To deal with this information they used keyword search engines, tagging schemes,
collaborative filtering or linguistic methods. These methods were not only costly and
time-inefficient but also non-scalable, inaccurate and taking the focus from core business.
80% of relevant information could not be utilized.
30 IDOL Server 7.3 Administration Guide Revision 7
33. 1 Introduction to IDOL Server
Autonomy Infrastructure
Autonomy's software infrastructure allows you to utilize 100% of the information that is
relevant to you. It automates all the business processes that formerly had to be dealt with
manually.
By developing a patented combination of Bayesian Inference, Shannon's information theory
and pattern matching, Autonomy has enabled computers to understand unstructured,
structured and semi-structured information. This means that Autonomy's software
infrastructure solves a fundamental problem that affects every industry, and can be used in
virtually any application that handles unstructured information:
e-commerce
CRM
knowledge management
business intelligence
enterprise information portals
online publishing
Autonomy's software infrastructure is fully scalable and allows you to process information:
automatically
in real time
in any language
IDOL Server 7.3 Administration Guide Revision 7 31
34. 1 Introduction to IDOL Server
Autonomy Infrastructure
Administration
IDOL Administration provides a distributed, Web-based infrastructure for managing IDOL
components and services. IDOL Administration is composed of:
Administrative Hub The Administrative Hub is a central configuration service that
maintains state information for all IDOL resources in a network. The
Hub communicates between the administration clients and the IDOL
services on various hosts. Also referred to as the Hub.
Administrative Nodes Each Administrative Node communicates between the Hub and the
IDOL services that reside on the host, such as IDOL server, DIH,
DAH, and so on. Also referred to as a Node.
IDOL Dashboard The IDOL Dashboard is a client of the Hub and provides a
browser-based interface that administrators use to install Nodes and
view component and service summaries.
Administrators use a sub-component of the IDOL Dashboard, called
the Dashboard Console, to administer Hub settings and to install,
configure and administer IDOL server and other Node services in the
IDOL network.
repository The Repository contains binaries for all IDOL components, the Node
installer, and the Dashboard Console.
Together, these components establish the IDOL Administrative Network. Refer to the IDOL
Administration User Guide for more information.
Note If you are using the Standalone version of IDOL, the administrative interface is
not provided, and you must manage components and services by editing the
configuration files directly.
IDOL Server
Using Autonomy connectors, Autonomy's IDOL server integrates unstructured,
semi-structured and structured information from multiple repositories through an
understanding of the content, delivering a real time environment in which operations across
applications and content are automated, removing all the manual processes involved in
getting the right information to the right people at the right time.
32 IDOL Server 7.3 Administration Guide Revision 7
35. 1 Introduction to IDOL Server
Autonomy Infrastructure
Connectors
Connectors enable automatic content aggregation from any type of local or remote repository
(for example, a database, a Web site, a real-time telephone conversation etc.), forming a
unified solution across all information assets within the organization.
Interfaces
Portlets are windows that can be set up in Autonomy's Portal-in-a-Box or third party
portals. Each portlet contains an application that allows the portals' end users to benefit
from a variety of IDOL server functionality.
Retina™ is an easy-to-use Web interface application that provides a full scale of retrieval
methods that adjust to the individual user’s proficiency.
Autonomy Desktop Suite™ brings the power of Autonomy to every desktop. Conducting
a real-time analysis of the ideas involved in the content of any opened desktop application,
Desktop Suite’s ActiveKnowledge or Active Windows Extensions module provides
real-time links to relevant internal and external information without the user being
needlessly diverted from his work in progress to perform an exasperating search or
retrieval operation.
Distributed systems
Autonomy’s distribution solutions facilitate linear scaling of systems through faster command
execution and reduction of processing time.
DAH™ (Distributed Action Handler) enables the distribution of ACI (Autonomy Content
Infrastructure) action commands to multiple Autonomy IDOL servers, providing failover and
load balancing.
DIH™ (Distributed Index Handler) enables distributed indexing of documents into multiple
Autonomy IDOL servers, providing failover and load balancing.
PODS
Autonomy’s Product Orientated Drop-in Solutions allow Autonomy solutions to be easily
integrated with third-party applications and solution providers. PODS enable organizations to
make their existing applications compatible with IDOL with minimal configuration and
IDOL Server 7.3 Administration Guide Revision 7 33
36. 1 Introduction to IDOL Server
IDOL Server Operations
administration requirements. Making IDOL server a part of any solution delivers the direct
benefits of content automation and the ability to perform a vast range of IDOL server
operations, irrelevant of file format or location.
IDOL Server Operations
Autonomy’s IDOL server can perform these intelligent operations across structured,
semi-structured and unstructured data.
Agents Alerting
Automatic Query Guidance Categorization
Channels Clustering
Collaboration Dynamic Clustering
Dynamic Thesaurus Eduction
Expertise Hyperlinking
Mailing Profiling
Retrieval Spelling Correction
Summarization Taxonomy Generation
Note Your license determines which of these operations your IDOL server
installation can perform.
Agents
Agents provide the facilities to find and monitor information from a configurable list of Internet
and Intranet sites, News Feeds, Chat Streams and internal repositories that is highly relevant
to the explicit interests of a user. Agents are created in a very user-friendly way using these
options:
natural language descriptions
example content (point and click)
legacy keyword or Boolean expressions
34 IDOL Server 7.3 Administration Guide Revision 7
37. 1 Introduction to IDOL Server
IDOL Server Operations
IDOL server provides the conceptual information that is needed to create agents. The server
accepts a piece of content (training text, a document or a set of documents) or reference
(identifier) and returns an encoded representation of the concepts, including each concept’s
specific underlying patterns of terms and associated probabilistic ratings.
Users can retrain their agents by submitting a piece of content (training text, a document or a
set of documents) whose concepts the server uses to adapt the agent.
Alerting
IDOL server analyzes data in new documents (when it receives the documents) and compares
the concepts in documents with users’ agents. If new data matches a user’s agent, it
immediately notifies the user by email or a third party system (for example by SMS or a
pager).
Automatic Query Guidance
IDOL server finds the most salient terms and phrases in a query’s results, automatically
clusters these terms and phrases and uses the clustered phrases to provide a hierarchical set
of queries that guide the user to the result area he is looking for.
Categorization
IDOL server can automatically categorize data with no requirement for manual input
whatsoever. The flexibility of Autonomy’s Categorization feature allows you to precisely derive
categories using concepts found within unstructured text. This ensures that all data is
classified in the correct context with the utmost accuracy. Autonomy’s Categorization feature
is a completely scalable solution capable of handling high volumes of information with extreme
accuracy and total consistency.
Rather than relying on rigid rule based category definitions such as Legacy Keyword and
Boolean Operators, Autonomy’s infrastructure relies on an elegant pattern matching process
based on concepts to categorize documents and automatically insert tag data sets, route
content or alert users to highly relevant information pertinent to the user’s profile.
This highly efficient process means that Autonomy is able to categorize upwards of four million
documents in 24 hours per CPU instance, that's approximately one document, every 25
milliseconds. Autonomy hooks into virtually all repositories and data formats respecting all
security and access entitlements, delivering complete reliability.
IDOL Server 7.3 Administration Guide Revision 7 35