Fixed couple bugs in query, and improved logic of querylib.
[moodle.git] / search / README.txt
0d46c846 12006/09/08
3Google Summer of Code is finished, spent a couple of weeks away from
4the project to think about it and also to take a break. Working on it
5now I discovered bugs in the query parser (now fixed), and I also
6un-convoluted the querylib logic (well slighlty).
8Updated ZFS files to latest SVN.
791a4cec 102006/08/21
12Fixed index document count, and created new config variable to store
13the size. (Search now has 3 global vars in $CFG, date, size and complete,
14see indexer.php for var names). Index size is cached to provide an always
15current value for the index - this is to take into account the fact that
16deleted documents are in fact not removed from the index, but instead just
17marked as deleted and not returned in search results. The actual document
18still features in the index, and skews sizes. When the index optimiser is
19completed in ZFS, then these deleted documents will be pruned, thus
20correctly modifying the index size.
22Additional commenting added.
24Query page logic very slightly modified to clean up GET string a bit (removed
25'p' variable).
27Add/delete functions added to other document types.
29A few TODO fields added to source, indicating changes still to come (or at
30least to be considered).
6f1b1da1 322006/08/16
34Add/delete/update cron functions finished - can be called seperately
35or all at once via cron.php.
37Document date field added to index and database summary.
39Some index db functionality abstracted out to indexlib.php - can
40use IndexDBControl class to add/del documents from database, and
41to make sure the db table is functioning.
43DB sql files changed to add some extra fields.
45Default 'simple' query modified to search title and author, as well
46as contents of document, to provide better results for users.
6e780562 482006/08/14
50First revision of the advanced search page completed. Functional,
51but needs a date search field still.
55Added resource search type, and the ability to specify custom 'virtual'
56models to search - allowing for non-module specific information to be
57indexed. Specify the extra search types to use in lib.php.
61Added delete logic to documents; the moodle database log is checked
62and any found delete events are used to remove the referenced documents
63from the database table and search index.
65Added database table name constant to lib.php, must change files using
66the static table name.
68Changed documents to use 'docid' instead of 'id' to reference the moodle
69instance id, since Zend Search adds it's own internal 'id' field. Noticed
70this whilst working on deletions.
72Added some additional fields to the permissions checking method, must still
73implement it though.
b585dc5f 752006/07/25
77Query logic moved into the SearchQuery class in querylib.php. Should be able
78to include this file in any page and run a query against the index (PHP 5
79checks must be added to those pages then, though).
81Index info can be retrieved using IndexInfo class in indexlib.php.
83Abstracted some stuff away, to reduce rendundancy and decrease the
84likelihood of errors. Improved the stats.php page to include some
85diagnostics for adminstrators.
87delete.php skeleton created for removing deleted documents from the
88index. cron.php will contain the logic for running delete.php,
89update.php and eventually add.php.
a25a3912 912006/07/11
93(Warning: It took me 1900 seconds to index the forum, go make coffee
b585dc5f 94whilst you wait.) [ forum data]
a25a3912 95
96Forum search functions changed to use 'get_recordset' instead of
97'get_records', for speed reasons. This provides a significant improvement,
98but indexing is still slow - getting data from the database and Zend's
99tokenising _seem_ to be the prime suspects at the moment.
101/search/tests/ added - index.php can be used to see which modules are
102ready to be included in the search index, and it informs you of any
103errors - should be a prerequisite for indexing.
105Search result pagination added to query.php, will default to 20 until
106an admin page for the search module is written.
110Search-enabling functions moved out've the mod's lib.php files and into
111/search/documents/mod_document.php - this requires the search module to
112operate without requiring modification of lib files.
114SearchDocument base class improved, and the way module documents extend
115it. A custom-data field has been added to allow modules to add any custom
116data they wish to be stored in the index - this field is serialised into
117the index as a binary field.
119Database field 'type' renamed to 'doctype' to match the renaming in the
120index, 'type' seems to be a reserved word in Lucene. Several index field
121names change to be more descriptive (cid -> course_id). URLs are now
122stored in the index, and don't have to be generated on the fly during
123display of query results.
abb4ea20 126------
127Started cleaning and standardising things.
129cvs v1.1
682d4032 131This is the initial release (prototype) of Moodle's new search module -
132so basically watch out for sharp edges.
134The structure has not been finalised, but this is what is working at the
135moment, when I start looking at other content to index, it will most likely
136change. I don't recommend trying to make your own content modules indexable,
137at least not until the whole flow is finalised. I will be implementing the
138functions needed to index all of the default content modules on Moodle, so
139expect that around mid-August.
141Wiki pages were my goal for this release, they can be indexed and searched,
142but not updated or deleted at this stage (was waiting for ZF 0.14 actually).
144I need to check the PostgreSQL sql file, I don't have a PG7 install lying
145around to test on, so the script is untested.
147To index for the first time, login as an admin user and browse to /search/index.php
148or /search/stats.php - there will be a message and a link telling you to go index.
150-- Michael Champanis (mchampan)
6f1b1da1 151 email:
152 skype: mchampan
682d4032 153 Summer of Code 2006