Hashemian Blog
Web Tools, Financial Markets, Technology
Monday, May 23, 2005
robots.txt
If you operate a public web site, there is little doubt that you'd like an occasional visit from search engine minions, known as robots. Robot are little agents that search engines dispatch to your site to scan your page contents and sent them back to the mother ship for cataloguing and finally including in search engine results pages (SERP).
If you have ever scanned your web logs, you would undoubtedly noticed these agents. They come with different names like googlebot, msnbot, and yahoo slurp. Almost all legitimate robots ask for permission before crawling a site, and the way that's done is through a file named robots.txt. This capability has been around since the early days of the search engines, but it is perhaps one of those often forgotten details. The reason is that if a robot can't locate /robots.txt on a Web site's root, it takes that as a green light to crawl and index the whole site.
robots.txt is flat ASCII file with a simple format. It is placed at the root directory of the Web site, so for example, it can be accessed this way: http://www.hashemian.com/robots.txt. If you want search engines to crawl your whole site, you would specify this inside robots.txt:
User-agent: *
Disallow:
If you want to block robots from a certain location of your site, you would specify this:
User-agent: *
Disallow: /certain-location
I won't bore you with the details. You can read about the stuff here.
Now the question is: if a missing robots.txt file is an open permission to crawl, why bother creating one? The best reason is to save on bandwidth. Many sites are designed to deliver a standard page to help lost users with missing pages. A robot looking for a missing /robots.txt file would also receive this page, and while in most instances, the standard error page will not cause any harm, the robot would still have to parse it, wasting bandwidth and resources. A safe practice to avoid this waste is to place an empty robots.txt on your Web site.
Finally, understand that /robots.txt works based on the honor system. While most legitimate search engines follow its instructions, there is no way to enforce obedience via this file. <robots.txt>
// posted by rh
Monday, May 16, 2005
Of French chocolate and wine
When it comes to chocolate, bitter is better, at least for me. I am partial to Swiss, but the French also make mean dark chocolates. Shopping at Trader Joe's today, my wife couldn't find my favorite Swiss dark, so instead she got me the French variety. You know, Chocolat Noir.
As it is my habit to read every possible word on a food label, I came across this sentence on the back of this particular chocolate: "Taste at room temperature like a good red wine". Only the French could come up with a phrase like that, comparing any food, or perhaps any experience, to wine. Now I know that as an American I am supposed to hate anything French, but I don't. I think Europeans know the real secret to life and how to enjoy it to the fullest.
My experience with the French culture hasn’t always been positive. I took seven years of intensive French courses in school and I resented the language throughout. All I can show for my French education is a handful of words I can barely pronounce. I can't count how many French exams I flunked, but they were many. In contrast, I loved the English language from the first time I was exposed to it. The unpleasant experience might have put me on the path of detesting anything French, but as I grew I learned that judging a culture by its language is hardly sane.
On my first business trip to France, I recall the chain-smoking French plant manager offering us a basket full of wine and beer during a meeting. As Americans we declined, but he was unabashed about enjoying a few during the meeting. Their approach and openness to the issues of human pleasures such as food and drink, tobacco, and sexuality are taboo subjects for our culture, yet I can't help being envious of their relaxed way of life.
I am certainly not all admiration for the French, nor am I planning to learn the language that tormented me so much as a child. But I'm unapologetic to admit that there are certain parts of their culture that we, as uptight people, can and should learn from. Swiss dark chocolate and Napa Valley red wine will do just fine. <Of French chocolate and wine>
// posted by rh
Wednesday, May 11, 2005
The case against human cloning
Even the most humble among us carry a little vanity picturing themselves as visionaries. So here's my little slice of personal prediction. I make the claim lightly with no presupposition of having delusional premonitions. It is just something I believe in.
The case for human cloning has been a hot subject for a while now. Conservatives are dead set against it, having a narrow definition of it known as procreation. Most liberals, on the other hand, are welcoming of the latest trends in cloning and embrace the possibilities of human cloning down the line. Most people wouldn't mind a shot at immortality. After all, if we are fundamentally happy with who we are, why not extend ourselves to infinity? I know all debates about death being an extension of life and God's will, but the next time you visit the doctor's office for some unexplained symptom, ask yourself why it has you so worried? The point is that most of us would rather extend our lives as long as possible, rather than face death.
Human body is a marvel of biology or a testament to the wisdom of God, if that definition works for you. But human body is essentially flawed. From the moment we are born, the forces of nature conspire against every cell in our bodies and there is no escaping the ultimate fate. Cloning could alleviate the condition, perhaps giving us a shot at longer lives. But why should we replace one buggy system with another one?
Enter technology. I know it sounds absurd, but technology has made giant leaps in unraveling the secrets of what makes us tick. Parallel to that is the exponential strides made in computer and robotic science. There is little doubt that in a not too distant future machines might equal or surpass our intelligence. Artificial intelligence is no longer the stuff of Hollywood. It is happening today, and it is improving at a rapid speed.
So, consider for a moment that at some point in time science and technology might give us the ability to transfer the essence of a human into a robot. I am talking about the wisdom, the consciousness, the very stuff that makes us sentient beings. If a machine can replicate all electrochemical activities that goes on within our skull cavities, then it might just be possible to transfer one's soul into a machine, replacing electrochemical reactions with electromechanical impulses. At such time there will no longer be a need for cloning, or the long shot at revival through cryogenics. People can continue life as robots or even self-deterministic programs running inside an elaborate machine, virtual reality plus.
It is perhaps with some uneasiness that I find myself on the side of conservatives with the cloning issue. Let God continue to dictate the biological definition of life. Perhaps the time will come when others might decide to diverge from the sacred dogma and chart a path to a parallel and co-existing definition of other forms of existence.
<The case against human cloning>
// posted by rh
Thursday, May 05, 2005
JavaScript void function and html hyperlinks
One of the techniques often used in html are hyperlinks that actually do not take the user to another page, but instead run a client-side javascript, dhtml.
The most common way of programming hyperlinks in html is done by using the A (anchor) tag. The following code snippet illustrates the ease of creating a hyperlink:
<a href="http://www.hashemian.com/">www.hashemian.com</a>
Since the A tag has been with us since the inception of the Web, most people have come to expect that clicking on them causes an action. And even the most novice of html programmers are familiar with the tag.
Things however can get more complicated if you need the hyperlink click do something other than redirecting the user to another page. For example, you want to pop up a message, populate a field, syntax-check the user's input, or other client-side tasks. You know, dynamic html (dhtml) stuff. Enter the javascript:void(0) function. I ran into this feature years ago while searching for something, and I've been a loyal user ever since. The javascript:void(0) function basically neuters the hyperlink while allowing you to carry out some dhtml activity within the onClick event, i.e.:
<a href="javascript:void(0)" onclick="alert('You clicked.')" >www.hashemian.com</a>
The void() function causes the browser to take no redirection action upon clicking. In most modern, javascript-literate browsers you can accomplish the same disarming task by adding a "return false;" at the end of the onClick event code, but you'd still need to provide something in for the href property, so might as well make it javascript:void(0).
The story of void() doesn't quite end here. Although javascript:void(0) is the most widely used form of the void() function, you can actually have it invoke a javascript function like so:
<a href="javascript:void(alert('You clicked.'))" >www.hashemian.com</a>
This obviates wiring up the onClick event altogether. As always, your mileage may vary depending on your browser, so test your pages thoroughly. <JavaScript void function and html hyperlinks>
// posted by rh
Wednesday, May 04, 2005
MSNBC.com's missing expandable menu
As a frequent reader of MSNBC's Web site, I started noticing this past weekend that their left-side navigation menu items no longer expanded. As of this writing, the menu has yet to regain its dynamic trait.
The expanding menu has been part of MSNBC's navigational feature for many years. As the user hovered over the different items, a submenu would branch off displaying links to the top news for that section and other relevant sub-sections within. The sub-section items, once hovered over, would in turn open up their own menus displaying relevant links.
I always liked this functionality. It provided a one-click access to the stories I wanted to view. Dynamic menus do come with some inherent issues. One of the most problematic is layering. Most dynamic menus have the unfortunate side-effect of being eclipsed by active controls on a Web browser. Those controls comprise items such as drop-down lists, applets, and Flash areas.
To solve that problem, MSNBC would hide the active controls on the page whenever the user hovered over a menu item, thus the expanded menu would not clash with other controls on the page. It meant that many times interactive banners would suddenly vanish, and I suspect the advertisers weren't so pleased about their banners doing the disappearing act.
Now, with the expandable menu gone (at least for now), MSNBC is reaping several benefits, albeit at the expense of upsetting the dynamic menu fans. The banners would no longer need to be hidden, the users who would need to click on and visit the various section pages to see the relevant links are now greeted with a sponsored splash page (read more page impressions), and MSNBC.com would no longer need to maintain the dynamic menu.
According to one of our Web designers, most people dislike dynamic menus because they interfere with the page and irritate the users. Perhaps that was part of MSNBC's reasoning to kill its dynamic menu. But given the other benefits, I hardly doubt MSNBC agonized much over this decision.
<MSNBC.com's missing expandable menu>
// posted by rh
Tuesday, May 03, 2005
Google vs. the dictionary and the encyclopedia
By the time I got home tonight, my 9-year old was nearly finished with her homework. The only question remaining on her assignment sheet was "What is Olympus Mons?"
Now I knew I had heard of this term before, but I just couldn't come up with a definitive answer. Was it a crater on the Moon? A rock formation on Mars? I was certain the term pertained to some off-earth object, but it's a big space with lots of objects.
So I promised her that we will look it up in the dictionary after dinner. Her response, "Let's look it up on Google first?" I was struck with how fast the Internet has endeared itself to even the elementary school kids today. The truth is that I do the same when I am looking for something, so why shouldn't she? But somehow I can't help feeling sad about how drastically the Web has mutated our culture.
Instead of opening a book or two, now we just Google it. In some ways we have been robbed from the fun and challenge of searching for something the old-fashioned way. But there is no defying progress.
As a compromise, I suggested for us to look the term up on wikipedia.com. At least that Web site has some semblance to a real encyclopedia. No dice, wikipedia was stumped, though it came up with some suggested links. But clicking on those would mean too much effort. And so Google became the clear the winner, and we didn't even have to click on any search results. In a flash of a page-load, the answer sat before us.
Olympus Mons, located on Mars, is the largest volcano in the solar system. A speedy answer, courtesy of the omniscient Google.
<Google vs. the dictionary and the encyclopedia>
// posted by rh
|
Links
Technorati Profile
TMCnet.com
ARCHIVES
09/01/2003 - 10/01/200303/01/2004 - 04/01/200404/01/2004 - 05/01/200405/01/2004 - 06/01/200406/01/2004 - 07/01/200407/01/2004 - 08/01/200408/01/2004 - 09/01/200409/01/2004 - 10/01/200410/01/2004 - 11/01/200411/01/2004 - 12/01/200412/01/2004 - 01/01/200501/01/2005 - 02/01/200502/01/2005 - 03/01/200503/01/2005 - 04/01/200504/01/2005 - 05/01/200505/01/2005 - 06/01/200506/01/2005 - 07/01/200507/01/2005 - 08/01/200508/01/2005 - 09/01/200509/01/2005 - 10/01/200510/01/2005 - 11/01/200511/01/2005 - 12/01/200512/01/2005 - 01/01/200601/01/2006 - 02/01/200602/01/2006 - 03/01/200603/01/2006 - 04/01/200604/01/2006 - 05/01/200605/01/2006 - 06/01/200606/01/2006 - 07/01/200607/01/2006 - 08/01/200608/01/2006 - 09/01/200609/01/2006 - 10/01/200610/01/2006 - 11/01/200611/01/2006 - 12/01/200612/01/2006 - 01/01/200701/01/2007 - 02/01/200702/01/2007 - 03/01/200703/01/2007 - 04/01/200704/01/2007 - 05/01/200705/01/2007 - 06/01/200706/01/2007 - 07/01/200707/01/2007 - 08/01/200708/01/2007 - 09/01/200709/01/2007 - 10/01/200710/01/2007 - 11/01/200711/01/2007 - 12/01/200712/01/2007 - 01/01/200801/01/2008 - 02/01/200802/01/2008 - 03/01/200803/01/2008 - 04/01/200804/01/2008 - 05/01/2008
|
