Forum:Differentiators with other search engines

[edit] Questions

During the initial phase it's a good idea to figure out feature differentiators with other search engines before any implementation details.

1. What's the vision of this social search engine project?
2. What makes it stand out at a higher level?
3. How wiki collaboration spirit can help to shape up its future?
4. How it can avoid being just another search catalog (compiled by human instead of machine)?
5. What will draw users to this new wiki search engine?
6. etc.

Brainstorm welcome. --75.6.242.254 19:52, 27 December 2006 (UTC)

[edit] Ideas/Answers

API - A really rich open API would be nice. One that doesn't limit access because of monitization worries. Possibly based around the OpenSearch protocol. -Steve 24.18.125.112 00:52, 28 December 2006 (UTC)

Structured Content - Design your engine from the ground up to be about cataloging and finding structured content. Scan the pages you crawl for Microformats and pull those into your index as first class items. Allow for the submission of other forms of structured content similar to Google Base. Use the API above to build a central repository of everything structured on the web. The community then becomes a voting machine for stack ranking this content. -Steve 24.18.125.112 01:03, 28 December 2006 (UTC)

There should be a API that allows extensions. One Extensions may handle the Microformat stuff. Personally it might be best to see a Microformat just as a way to gather RDF out of an xHTML Page. Therefore, while we are searching Webpages containing a Microformat, the search itself needs best to be based on RDF. MovGP0 23:57, 29 December 2006 (UTC)

Topic Guides - One way to put a Wiki like spin on things would be to allow the creating of topic or category guides that surface like instant answers. If every crawled page were to get categorized into a bucket or two you could simply analize the top 10 results, find the dominant category if any, and surface the appropriate guide for the predicted category. If I search for "camcorders" I might get a guide that gives a quick overview of camcorders and offers suggestions for how to refine my search. I don't think you want these entries to become a directory though as that might show bias. -Steve 24.18.125.112 05:23, 28 December 2006 (UTC)

Semantic Network - Similar to Topic Guides mentioned above but flexible enough to dynamically disambiguate search queries to improve search quality. Search index can be built in association with this Semantic Network, parsing both the web and the query for better understanding at semantic level and match. This is not practically available from current major commercial search engines. - Dan January 10, 2007.

... —The preceding unsigned comment was added by 67.120.87.215 (talk • contribs).

Simple Query Editor

Caching semantic data within relational tables

If you are interrested in semantics than consider this: Forum:Idea for semi-automatic semantic tagging.

Anyway I'm very impressed by Google Base. I think it should be possible to define a database on-the-fly to add such structured data. The Wikidata Project is currently implementing such a thing. Also defining Semantics should be possible in an interactive way like SWOOP does.

Anyway, I see some problems with this when we try to combine semantic and relational data, because a relational Database is faster, but less flexible than a semantic Database.

I think we could solve this by letting the users define a "view" of the semantic database which represents a part of the semantic database as relational database, so we get the best of both worlds. I think such a solution could work similar to Ontoworld's Inline queries, but caching the result in a separate, dynamically created, Database.

As you can see in the image, I'm also storing a token within the cache, so that we can do proper mapping from the relational store to the semantic store, when the user is doing edits (updates, inserts, and/or deletes) on the relational data.

— MovGP0 09:39, 10 January 2007 (UTC)

This brings up a good discussion thread: How can the search index to be built in association with semantics? Implementation wise what's the best way to organize the search index which may or may not be in the form of relational data. Look forward to more input along this line. — WikiCloud January 17, 2007