So what is it? It’s called Google Co-op, a platform which enables users to build their own vertical search engines and make money off the advertisements. It provides a clean, easy interface for simple site restrictions (like what Yahoo! Search Builder and Live Macros offer) plus a number of power user features for tweaking the search results. The user has control over the look and feel (to embed the search box on their own site), can rank results, and even (multi) tag sites to let viewers filter out results by category.
But talk is cheap. So let me show you some examples of what you can do with Co-op:
http://vik.singh.googlepages.com/techstuff
This is a technology specific search engine, which lets users refine results based off Google Topics (global labels which anyone can annotate with). Basically, I was lazy here. I didn’t feel like multi-tagging sites/domains individually, so instead I just collected a laundry list of popular technology site domains in a flat file and pasted it into Google Co-op’s Custom Search Engine control panel/sites page. In addition, something I think is really useful, Google Co-op allows users to bulk upload links from OPML files. So, to make my life easier when building this, I uploaded Scoble’s and Matt Cutt’s OPML’s. Tons of great links there (and close to 1000 total). Then I clicked on the ‘filter results to just the sites I listed’ option (which I recommend you use since if you muddle your results with normal Google web search’s you typically won’t see your results popping up on the first page of results despite the higher priority level promise for hand chosen sites). To enable the filters you see on the results page (Reviews, Forums, Shopping, Blogs, etc.), I did an intersection with the background label of my search engine and the Google Topics labels. How do you that? The XML context configuration exposes a <BackgroundLabels> tag. Any labels listed in the BackgroundLabels block will be AND’ed (how cool is that). So I added the label of my search engine (each search engine has a unique background label – it can be found bolded on the Advanced Tab page) and a Google Topic label (News, Reviews, Stores, Shopping_Comparison, Blogs, Forums, etc.) in the BackgroundLabels XML block. I made a separate XML context file for each Google Topic intersection. By doing this, I didn’t have to tag any of my results and was still able to provide search filters. Google Topics does most of the hardwork and gives me search refinements for free!
But say you’re not lazy. Here’s an example of what you can do with multi-tagging and refinements.
http://vik.singh.googlepages.com/machinelearningsearch2
This one is more of a power user example – notice the refinements onebox on the search results page, and the labels with “>>” at the end. These labels redirect to another label hierarchy (a hack, I used the label redirect XML option to link to other custom search engine contexts – basically I’m nesting search engines here)
Now, say you want to get fancy with the search results presentation. Here’s a way to do it with Google’s Ajax Search API:
http://www.google.com/uds/samples/cse/index.html
Thanks to Mark Lucovsky and Matt Wytock for developing that great example.
For more information about how to use the Ajax Search API with Custom Search, please take a look at this informative post: http://googleajaxsearchapi.blogspot.com/2006/10/custom-search-engine-support.html
While writing this blog post, I realized it would take me forever to go over the number of tricks one can pull with Co-op. Instead, I’ll summarize some of the big selling point features to encourage everyone to start hacking away. Also, to help jump start power users, I’ve linked the XML files I used to make my featured search examples at the bottom of this post.
Key Feature Summary (in no particular order):
- Make money (get a share off the ad clicks)
- Multi-label sites
- Have up to 5000 annotations
- Can collaborate with friends to tag sites (I’ve made my search engines public so anyone can add their annotations)
- Can associate weights to results and have control over the rankings (refer to the <score> tag in the XML)
- Completely brand the engine and customize the look and feel
- Can display refinements on the search results page
- Can upload XML/OPML files filled with links
- Can upload XML to configure the search engine labels/contexts
- Can even nest label hierarchies on the refinements onebox
- Ex. http://vik.singh.googlepages.com/machinelearningsearch2 .. click on Sources
- Can make the search Ajax’y
- Can combine results with another person’s search results
- (by intersecting the background labels in the XML advanced configuration file)
and much much more (especially for power users).
If you need a search engine for your site, and your content has been indexed by Google, then seriously consider using this rather than building your own index – or worse, using the crappy full-text functions available in relational databases.
Here are my XML files:
tech-stuff-scoble-annotations.xml
tech-stuff-matcutts-annotations.xml
Happy Coop hacking!
Vik's Blog 
ok, looks wonderful, but the limited page width of googlepages does not allow the ads to appear… (same on my trial, http://gtziralis.googlepages.com/search)
any ideas on that?
thanx in advance
In your ajax example, where in the code would you put your adsense info?
Hi George – Google is working on fixing that issue. Currently the size of the iframe is hardcoded but they plan on making it resize dynamically to adjust for different page layouts. For now, try going to the Code tab and selecting Top and Bottom for the ads presentation option.
Vik,
great work. I was reading your post on Techcrunch and I thought you came to the table with some real value. I gave your AI search engine a spin, very impressive. Regards
looks some good and interesting work here
Seems really cool. But how far can you edit? What does Google profit on this`?
The Custom search engine pages on Google allow some limited customizing colour etc, but for more advanced customising try using the Google AJAX search combined with Custom search engine I have put a quick how to on our blog to help people get started.
http://ventureskills.wordpress.com/2006/10/26/google-custom-search-engine/
Google have also put up a couple of examples on there site, to customise the output you need to download the css which I don’t go into on that post.
Tim
In The Stromboli Project we´ve prepaired an erotic stories search engine!
That´s in http://searcheroticstories.thestromboliproject.com
Enjoy it!
😉
How does one embed the search results in a worpdress site?
Vik,
Thanks for sharing and explaing some of the features.
i am trying to re-design the refine div,
is that possible like in the techstuff page?
Allyn – if your looking to embed in wordpress.com site, you will find it very difficult if not impossible but on your own instance of wordpress, you can easily add it as a widget or directly into the template.
Sid – what do you mean refine the div? if your using the ajax search to intergrate it into your own site copy googles css from there site onto your own and make changes to that.
Great Article! I just have one question: It is possible to refine the search before the results appears? I mean if the search box can be tunning to perform the search in one (or several, but not all) refinement categories?
I am looking the answer of this, but just cant find it…
Thank you!
First
Thakns ventureskills for responding to the previous two questions
Guillermo – Yes you can refine from the get go. A couple of ways of doing this:
(1) Treat each refinement as a separate search context. Then just put their background labels in the block in the xml context for your primary search.
(2) Append “more:” operators to the user’s search query.
(3) Actually, just check the advanced documentation under Background Labels. You can place refinement labels in the block directly without having to abstract each one as a separate search.
Thank you so much, zooie!!! For the great answer and how fast you reply. I will look for what you sugest. Thanks!!!!
Ok, the first two methods definitely will work. The third method sounds very nice, but I only find information to use the background labels in the xml to configure the site, not to perform the search… Is that correct?. Actually I think that the question should be: It is possibly to specify the refine labels in the code of the search box? Is that can be done with the background labels in the javascript code, i didnt find the way… Sorry for bother you…
http://www.google.com/coop/profile?user=015417630718833783855
I would like to combine all my search engines into one “meta” search engine for stained glass.
How can I do that?
Guillermo – I’m not sure what you’re referring to by the ‘javascript code’. Refer to the advanced documentation that describes backgrounds labels (the XML context). If for some reason that’s not working for you, then try the first two options (the second being the easiest, just append ‘more:” to the search query).
Stellula – Hmm … there are a couple of ways. First is to redo your annotations and add them to just one search engine context (you just need to download the annotations in XML format from the Advanced tab and the change the context string so that it references the primary search context). Another way is to provide all these search engines as refinements (like I did in techsearch). You can create refinements for each meta search engine which label redirects (the label here should be the background label string specified in the advanced tab for that particular search engine).
My Google Co-op example:
You Search for unprotected live webcam streams found through a variety of clever search techniques done with the Google Co-op custom search engine tool.
http://www.camhacker.com
Hi everyone.
We created a free directory for your cse at http://www.coopdir.com
Only cse created with google co-op will be accepted.
Thanks
ps Nice tips.
Vik/Zooie, your post is an absolute treasure trove of great CSE info….
I don’t have the tech skills to pull 95% of this XML or Ajax stuff off…. but if I get the time to try, the ideas, examples, and files you’ve posted here will be critical….
Many thanks…. Your search engines and website rock.
Here’s my CSE Attempt, by the way:
Film & Performing Arts
http://google.com/coop/cse?cx=012633772325726105531%3Avkzo3156diy
(I’m also planning on adding nested refinement labels, eventually, as well as related Google Base, Blog, and News integration, although it’s likely way beyond my technical skill…)
cheers,
Luke
Writer/Editor, NYC
Guys, what are the possibilities to tweak Google CSE programmatically ? Can I create CSE using any API ? Can I programmatically manage list of sites, indexed by specific CSE?
I know that all this stuff can be done on Google site using UI, but I need that to be done on my site programmatically.
Svyatoslav – Yes. Refer to the advanced documentation. You can specify your refinements and annotations in XML (just use a program to write that) and upload them accordingly in the Advanced tab. Also, stay tuned for more support.
Hi Vik,
Thanks for this fabulous information. I tried to incorporate Google co-op, it worked fine for couple of days on my website but just stopped working after 5 days. I mean its there on my website, configured to display results from my website but it wont display the results anymore but just ads.
It used to give me results for the same query from within my website but it does does not anymore. At most what it returns is the home page URL. Pretty strange isnt it. Need your advice please.
Just to let you know, I had made a lot many changes in just 3-4 days within my website. In title tag and even in the body just to practice some SEO stuff. Could you please help me out.
Many thanks,
Chandan
Hi Chandan – My advice is to post your issue on the Google Co-op discussion forum (they are _very_ good at responding quickly to matters like these):
http://groups.google.com/group/google-co-op/topics
Tell me if they help. If not, email me and I’ll work with you through the steps.
Hi. I just set up a Google custom search at http://www.boatr.org.uk for the boating community. It took far less time to set up the website and incorporate Google co-op, adsense and analytics than it did to do the research into which 200+ websites to incorporate. I’m getting some good feedback from the users and hope to play around more soon. Thanks for the ideas!
Is there a way to customize COOP to display results on your site without showing Google Ads?
I think if you’re a special non-profit group you can avoid displaying advertisements. You can check out the TOS or better just email the Co-op discussion forum to see if you can get leeway for this.
Nice articles you have on your site. Just wondering about the strength of google COOP and decided to create a web search site with it and wanted to share this with you and your users about how interesting and powerful the google COOP is.
New Google CSE searchengine lauched! SearchHacker, Quick searches for many different file types like:
mp3. mid, wav, au, wmv, avi, mpg, mov, mpeg, doc, xls, pds, txt, jpg, rar, torrent… you name it, can come in very handy! Have a look: http://www.searchhacker.com
Check this out http://www.googlepowersearch.com.
I created GooglePowerSearch so you can power search for Video, News, Maps, Images and more…
Google Power Search helps to unleash the built in power of Googles special features.
Using Google Power Search you are able to get better-targeted results.
Check out Google Power Search and let me know what you think.
Thanks
Steve
Checkout http://www.unitedtorrents.com – The power of google for all your torrent searches
Searching 1200+ torrent websites with over 6,000,000 torrents indexed. The ultimate torrent search engine using google co-op.
Looking for ebooks?
http://ebooksearchr.googlepages.com/ searches more than 100 ebook sites with direct download links available.
Man, let me tell you! I was looking for a great and easy way to have a search on my site. I went to GOOGLE after tons of messing around with lamer scripts. So, used the co-op thingy…see what it could do but wanted more options. In seeing what you did like removing some if not all of the links placed by google. See, I don’t want to remove any google copyrights but was pondering over the thought if I could in fact add my own links to my custom search? That’d suit me fine. Thanks again for all this great reading and examples.
>>If you need a search engine for your site, and your content has been indexed by Google, then seriously consider using this rather than building your own index
We seem to have a problem getting indexed and re-indexed by Google fast enough to get a good use of Co-Op as our own site search.
Anybody knows some secrets how to get indexed fast enough so we could use Co-Op on the site with user generated content?
Thanks.
Unfortunately there’s not much you can do here. You can fiddle with Google Sitemaps to hint to Google when the site needs to be crawled again, but there are no guarantees. Co-op, just like normal web search, doesn’t do real-time indexing. You may want to look into server side search solutions (such as those crappy database full-text plug-ins). The rule of thumb is if you’re searching through lots of different sites, already indexed by Google, that don’t typically change often, then you should opt for Co-op. If you’re requirements are different, then you should look into server side search.
Thanks.
Vik,
I really like your CSE. How I can I fix the search to have it automatically return a keyword search, when someone clicks on a link to a site after doing a search on the net?
Example: You search for golf clubs, ABC Golf is in your search results. You click the listing and the CSE generates the listing according to the keyword.
Any suggestions?
Thanks,
Rod
Here are some more hacks for google
http://seomization.blogspot.com/2007/09/google-hacks.html
Vik,
I love your implementation. But I’m trying to figure out — are these your Adsense ads displayed above and below the results? I don’t see what in the code links to your AdSense? And if not, then I’m assuming these are just google’s own Adsense ads…. is there then a way to modify this to act like Google’s Site Search for Adsense, where you get a cut of the ad clicks in the search results, but with the flexibility of your CSE implementation? I know that one can do this with the standard CSE implementation, but that involves using Google’s own code.
thank you vik for the hacks and the codes. Very usefull. I am running a coop by myself since 2006 ( http://www.lukas119.de ) and i am very excited that coop rocks. Thanks again for the CSE xml examples above