Hashemian Blog Web Tools, Financial Markets, Technology Wednesday, August 27, 2008 Online Car Registration Renewal, Finally  It's taken a long time. Perhaps a decade late for the Connecticut Department of Motor Vehicles to carry out some transactions online, but better late than never. While most of the private industry has shifted their operations online, the government is painfully behind on that front. The reasons range from the government being generally slow, to having under-paid technology workers and to shunning change. But change is inevitable and the DMV has finally taken some steps to that end. When I received my registration renewal papers in the mail a couple of weeks ago as I have for years, I started to reach for my dusty checkbook to write a check and mail it in. that's when I noticed a new section on the paperwork detailing their online renewal service. I thought this was one of those payment processing sites that handle credit card payments on behalf of the government offices and charge an extra fee. But no, this was the real deal. An actual DMV site that handled renewals and accepted credit cards with no extra fees. It took a mere couple of minutes to take care of the business on the site and I had an email confirmation soon after. The renewal document and sticker followed about a week later. It took them quite a long time to get on with the technology but I'm happy that they finally made it. I just hope that enough people make use of it and there are no major glitches to force them back to the old way. Online payment sure beats writing checks and licking stamps. dmv,car registration,online payments Labels: cars, web <Online Car Registration Renewal, Finally> // posted by rh 0 comments Saturday, July 19, 2008 Yahoo Slurp Versus Googlebot  As a part of my early morning routine at my job I scan a daily notification on our web site visitor patterns, specifically looking for abusive or abnormal page requests to identify rogue hosts. I wrote the program years ago, but it has become one of my most effective tools in identifying abuse. No surprise that search engine robots (spiders or crawlers) always top the list. Spiders are necessary evils. You want them crawling and indexing all your pages frequently, but that leads to spiders with voracious appetites that can rob the infrastructure of performance. Speaking of voracious appetites, for years Googlebot used to top our crawl volume list, far and away from the second position, usually (but not always) occupied by Yahoo Slurp (Yahoo's spider). As of a few weeks ago I am noticing that Yahoo Slurp crawl volume is surpassing that of Googlebot. The first few days I considered it an anomaly, but the levels have remained consistent, knocking Googlebot out of its first position. Googlebot still registers with more or less the same numbers, but Yahoo Slurp has been consistently beating those numbers, sometimes by twice as many. That's a considerable spike in Yahoo's crawl activity. I find it interesting that Yahoo's crawl volume started to rise in the midst of its take-over battles with Microsoft. Search engines have always competed on who has the most indexed pages. It's a bragging right, more than it is a practical or useful matter in search accuracy. But accuracy in search results is subjective, while page volumes are concrete numbers and that's why they are sometimes used as key differentiators between competing search engines. Still, the timing of the jump in Yahoo's crawling activity is intriguing. Could the motive be to enhance its index size (considered a key asset), thereby adding to the company's value? One wonders. Meanwhile, sites need to consider the battle repercussions between the search engine titans as their increased crawl rates could put additional pressure on their infrastructures. One way to mitigate the effects is to set up mirror servers and deflect the spider/robot traffic to them using application-aware appliances, but that there's some effort involved and there's room for error as the servers need to be synchronized in real-time, specially for those such as news-related sites that continuously publish new pages and fresh content. The other consideration is bandwidth management to accommodate increased traffic. At the least, faster servers and fatter pipes are in order. As a reference here are the current signatures left behind by Google and Yahoo crawlers: Googlebot's user-agent: Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) Yahoo Slurp's user-agent: Mozilla/5.0+(compatible;+Yahoo!+Slurp;+http://help.yahoo.com/help/us/ysearch/slurp) google,yahoo,search engines,http,seo,googlebot,web Labels: web <Yahoo Slurp Versus Googlebot> // posted by rh 0 comments Wednesday, June 18, 2008 DNS Mystery, NameServers, IP addresses  Today I was trying to reach 1&1's home page, but the browser kept failing to pull up the site. Mysteriously I was able to reach 1&1's home page when I changed my DNS servers to those of OpenDNS.org. Feeling curious I decided to investigate the matter in depth. My default DNS server was reporting the IP address of www.1and1.com to be 217.160.232.1. While that address belongs to 1&1, it's really one of their routers or gateways and not a Web server. No wonder I was unable to access the site. the working IP address reported by OpenDNS.org and a number of other DNS servers was 217.160.226.203. That is indeed the correct IP address for www.1and1.com. So why was I seeing different results from different DNS servers? As you may know the job of translating a host name to an IP address falls on a program known as the resolver which queries its designated DNS server for the answer. If the DNS server can not produce the translation (from its cache or authority zone), it issues what it's know as a recursive query to the DNS network on the Internet. The host name is broken to its fragments and each fragment from right to left is queried successively. The results generally consists of hosts known as NameServers, which get the query one step closer to the final result. The final NameServers produce the IP address translation. However, if any of the NameServers along the way can produce the translation, the query stops and the IP address is sent back to the resolver. Using the Unix/Linux dig command I followed the name resolution for www.1and1.com one step at a time. Results are shown here and shortened for brevity. This command displays the root servers: # dig ;; ANSWER SECTION: . 451081 IN NS M.ROOT-SERVERS.NET. . 451081 IN NS A.ROOT-SERVERS.NET. . 451081 IN NS B.ROOT-SERVERS.NET. . 451081 IN NS C.ROOT-SERVERS.NET. This command queries one of the root servers and produces NameServers for "com." TLD (Top Level Domain): # dig +norec @A.ROOT-SERVERS.NET www.1and1.com ;; AUTHORITY SECTION: com. 172800 IN NS K.GTLD-SERVERS.NET. com. 172800 IN NS L.GTLD-SERVERS.NET. com. 172800 IN NS M.GTLD-SERVERS.NET. com. 172800 IN NS A.GTLD-SERVERS.NET. This command queries one of the "com." NameServers: # dig +norec @A.GTLD-SERVERS.NET www.1and1.com ;; ANSWER SECTION: www.1and1.com. 172800 IN A 217.160.232.1 ;; AUTHORITY SECTION: 1and1.com. 172800 IN NS ns27.1and1.com. 1and1.com. 172800 IN NS ns28.1and1.com. Generally the previous command shouldn't produce and IP address, instead the authority section would prompt a final query to one of the 1and1.com NameServers (which by the way have the correct IP translation.) Instead somehow an IP address is produced at this level and the query ends with this inaccurate IP translation. I've tried the same query with the homepage URL's of Microsoft, Google, Yahoo and a few other sites and none return an IP address at this level. It remains to be seen if this erroneous translation would eventually spread around, causing 1&1's homepage to become widely inaccessible. Anyone knows how that IP translation ended up in of the "com." NameServers? Am I making wrong assumptions here? Feel free to let me know. dns,nameservers,domain names,ip addresses,internet Labels: internet, networks, web <DNS Mystery, NameServers, IP addresses> // posted by rh 1 comments Thursday, May 01, 2008 Google Image Labeler  Here comes Google with yet another Beta version of a product. Only this one is like a game with a fun twist and could get quite addictive. The object of the program is to use human intelligence to label images. As powerful and ubiquitous as computers have become, there are still many tasks that us humans are still more skilled at. In this case, identifying a photo or an image (especially a blurry or a vague one) is a task best left to the human brain. Amazon has capitalized on the same concept with its Mechanical Turk site. In that site people create tasks, called HITs (Human Intelligence Tasks) and invite others to respond. They could be research quizzes, surveys, categorizing web sites, or writing articles. Responders are paid for successfully finishing the tasks and Amazon keeps a commission. In Google Image Labeler, Google harnesses its vast visitor pool to assign labels to images. Two people are paired at random for each round and for 2 minutes are shown random images in sequence. Participants are tasked with coming up with as many labels as they can for each image. One side doesn't see the other side's suggestions. If one of the labels match, the participants are given a score and they move on to the next image. Or they can skip the image. What's in it for the participants? A journey into the psyches of 2 randomly connected people for 2 minutes at a time, and accumulating scores, perhaps for bragging rights. And for Google? A cost-free experiment to more accurately identify the images in its vast database. Since participants don't know each other and time is short, they are motivated to quickly suggest the most appropriate labels based on their visceral reactions. If you get a chance, give Google Image Labeler a shot. Just be warned that it could get addictive. I had to stop myself after a few rounds, lest I waste hours in oblivion. Speaking of Google, the stock climbed another 19 points or 3.25% percent today to $593. Since its low of $413 on March 10 (barely 7 weeks ago), it has risen nearly 37%. Could've, should've, would've. google,amazon,images,human intelligence,mechanical turk,psychology Labels: amazon, google, web <Google Image Labeler> // posted by rh 0 comments Tuesday, April 01, 2008 Citibank Online Banking?  If it was an April fool's joke, surely many people didn't find it amusing. While the stock market was rocketing up today, Citibank's online banking site was down and out, stranding thousands of their customers. We're not talking about a short outage, this was an all day near blackout. What astounds me is that there was no admission and no warnings on the site indicating the problem. It was as if everything was running smoothly. Could it be that the people in charge didn't even know about the problem for hours, or perhaps they were too arrogant or too cowardly to admit the issue and warn their customers? Meanwhile many people who have come to rely on Citibank's online banking, no less at the encouragement of the bank itself, were shut out without any means of conducting their businesses. I can understand that computers and Web sites go on the fritz sometimes. That's what redundancy and failover is for. Short of poor planning, Citibank could have at least had the decency to notify its customers of the outage and suggest alternatives. Instead it decided to bury its head in the sand and not even promptly respond to customer inquiries. Online banking should no longer be considered a luxury, it's a necessity to many people. This is not a site about the latest escapades of Paris Hilton or Britney Spears. It's an integral part of many people's daily lives. With all the problems that's facing financial institutions these days, this is a sure way for a bank to send its customers rushing for the exit doors. citibank,online banking Labels: business-finance, web <Citibank Online Banking?> // posted by rh 0 comments Wednesday, March 19, 2008 Google, Blogger in German?  Here's a head-scratcher for tonight. When I got home one of my kids was trying show me something on blogger.com and suddenly everything was in Deutsch, as in German. At first I though that it was a mix-up. Some programmer at Google (which owns Blogger) had screwed up and suddenly everything had gone Teutonic. Now we have a German-speaking household, so then I assumed that the kids had changed some setting to German language and Google was happily obliging. Hoping to unravel the mystery, I logged in to my own account on the XP machine and headed to google.com. no luck, it kept switching over to google.de. What was happening here? I checked the regional settings on the XP machine and everything was as expected, en-US. Then I headed to the whoami page to see if my IE browser was specifying a wrong "Accept-Language" header (i.e. de-DE) prompting Google to redirect to its German site based on the CultureInfo. Nope, that wasn't the problem either. Then I launched Firefox and surfed onto google.com. Same behavior, I landed on Google's German site again. Somehow Google was convinced that I was German and it was trying to help me, more like coerce me, to their German site. Is it possible that they had an algorithm tracking web sites visited from my house and deciding that the "most appropriate" site for us was their German version? I can't say for sure, but apparently Google had decided that we should use their German site. A quick fix was to click on the English link on Google's homepage. That deposited a cookie in the browser indicating that I was interested in the standard google.com site and it fixed the immediate problem, but not entirely. Blogger was (and still is) coming up in German and deleting cookies, as I often do, would bring things back to the annoying redirect. Was Google erroneously identifying my IP address as one from Germany? Feeling frustrated, I searched the newsgroups for an answer and I found this thread that confirmed my suspicion. One of the posters who was experiencing the same problem had written to Google and had received this response: Google has recently started using IP-address detection to help our users find our foreign destination sites. Unfortunately, our IP-address detection is not perfect and you are being inappropriately redirected. We are working on the problem. In the interim, you can regain your old Google.com experience by simply clicking on the 'Google.com [English]' link in the footer of the page. By clicking on this, Google will note that you have opted out of the foreign domain site and you will no longer experience redirects. So it was an IP address mis-identification after all. This is the most ridiculous scheme I have heard of, and to have it come out of Google, it is almost inconceivable. No doubt the folks at Google are padding themselves on their backs for their ingenuity. Meanwhile they apparently have disregarded the basic rules of IP addresses on the Internet and the proper way of serving visitors. First off, please don't help me along when I haven't asked for it. That is so Microsoft to try to be helpful in all the wrong places. If I want the German site, I'll ask for it myself. Second, the correct way of determining a user's preferred CultureInfo is checking the browser header. The browser will tell you what language the user is likely interested in. Third, the IP address is an unreliable way of determining the user's preferred language. The user could be an American connecting his laptop from his hotel room in Germany. Or a Parisian employee of an American company using a corporate proxy server in the U.S. Fourth, if you must persist in this IP detection insanity, at least make sure your databases are not flawed. I'm connected over an AT&T DSL circuit in Connecticut, yet Google apparently believes I'm in Germany or Lichtenstein. And finally don't give the visitor some half-baked solution of clicking on a link, when it doesn't help with your other sites and reverts to the original problem when cookies are deleted. Not to mention that it wouldn't even work if the user has opted to block cookies. For a company that prides itself in its technical prowess and its service, Google sure astounded me with this one. Or, is this just a ploy to throw some traffic to their foreign sites? Google,German,ip addresses,internet,web browsers,Firefox,cultureinfo Labels: google, internet, web, web browsers <Google, Blogger in German?> // posted by rh 0 comments Sunday, February 03, 2008 Microsoft + Yahoo > Google?  For those of us who might have thought that Microsoft's acquisition of Yahoo was an ace in the hole, this blog post from a Google bigwig might give a pause. Could Google be joining the likes of IBM, SUN, RealNetworks, Borland, Novell and Netscape who've lodged anti-trust complaints in various regulatory bodies around the world against Microsoft? Alright, no tears for Microsoft here. We all know this company is predatory and brutal when it wants to subjugate competitors. But can Google with a 75% share in their market (online search) really have a valid complaint here? Apparently so, and I'm actually surprised that Google has even addressed this acquisition rather than giving its characteristic aloof response. With Google's market value markedly below its 52-week high and facing slowdown or saturation in some of its markets, I can understand why the giant is suddenly feeling worried about its prospects. Can this acquisition finally give Microsoft the needed ammunition to meaningfully challenge Google? Only time will tell, but the fact that Google is feeling uneasy about it promises some interesting jousting and parrying ahead. Whatever the case, I hope Google doesn't lose its grip and mire itself in a long battle with Microsoft. Instead it should just stick to its guns, do no evil, and continue to innovate around Microsoft. In the end Google may still get run over by the behemoth, but I really hope Microsoft doesn't win this match, with or without yahoo. It will be a dark day on the Internet if Microsoft strips Google of its status just by its monopolistic tactics. google,microsoft,yahoo,search engines,anti-trust,acquisition,hostile takeover Labels: business-finance, google, Microsoft, web, yahoo <Microsoft + Yahoo > Google?> // posted by rh 0 comments Saturday, February 02, 2008 Face Cream Gimmick  Microsoft's $45 billion offer to buy Yahoo has certainly intensified the online advertising scrutiny. No doubt the entire advertising industry is going through turbulent times. At $20 billion per year, online advertising is still a small fraction of the entire advertising market, but that figure is estimated to rise sharply as more people turn to the Internet for their news, entertainment, and other personal and business matters. Indeed the cyberspace is no more immune to false advertising than other traditional methods. There are plenty of these online gimmicks around, many appearing on even reputable sites. From cars, to mortgages, to medical and beauty products, they make claims that are nothing short of miracles. And I suppose they sell well, because they seem to be everywhere. For example, this is a before and after shot of a woman's face on an ad banner touting some miracle cream to recapture youth. I keep seeing this over and over on msnbc.com. Is this an instance of false advertising? You decide. face cream,online advertising,banners,yahoo,microsoft Labels: advertising, health, Microsoft, web, yahoo <Face Cream Gimmick> // posted by rh 3 comments Wednesday, December 12, 2007 Google Lifeblood  Like most people who have a Web site I check my site's ranking on Google SERPs (Search Engine Results Pages) from time to time. It's striking how much of a Website's life depends on Google. That's particularly true with smaller sites whose lifeblood is the traffic Google sends their way. But even bigger sites would suffer severely if their pages suddenly lost ranking in Google. Sure there are other search engines like Yahoo and MSN, but enough about those. And so when a couple of days ago I noticed that my site's traffic had a noticeable drop in traffic, the first place I looked for diagnosis was Google. Sure enough, my site's pages where either non-existent or had dropped considerably in ranking. I know that compared to other sites, my traffic is but a drop in a proverbial bucket, but even so the realization of lost ranking made me concerned. I can only imagine how those people, whose living is tied to their traffic, may feel when Google starts to snub their sites. the results could be devastating. Had I violated any one of Google's quality guidelines? Had I engaged in any activity that might have blacklisted my site? I was stumped. I hadn't made any design changes to the site that I could recall. I even tested my site for unintended search engine spamming using a couple of different online tools. One claimed I had hidden text on my pages. They were light-colored timestamps on a colored background. Just for insurance I changed them to a darker color. It also caught what it regarded as keyword stuffing. The culprit turned out to be whitespace characters (&nbsp;) with missing trailing semi-colons. So at least I got to fix this error on my site, and then I just moved on. Today, inexplicably my site's ranking in Google SERPs seems to be back where it used to be. Could this have been the result of those minor changes? I don't think so. Most likely, the drop was due to some temporary event in Google's algorithm. What's alarming is that Google is not just influential, but it's vital to so many. Where can one go to if they are unfairly treated? Who will listen? This is not a paid service, there are no SLAs (Service Level Agreements), contracts, or even tenuous promises. Mine is just a hobby site. Being present in Google is great, but I'd still be doing this even if my site wasn't included. I don't think my attitude would be the same if I were making a living off my site. I can appreciate that Google has the enormous task of separating the good sites from the bad. But with that much power and reach, it is inevitable that many innocent sites will be inadvertently punished. Consider how things would be if there were only one powerful and unregulated credit agency with two marginal ones, instead of the three with equal standings today. google,search engines,seo,serp,spam,page ranking Labels: google, seo, spam, web <Google Lifeblood> // posted by rh 0 comments Monday, November 26, 2007 RSS ATOM Widget Utility  For the longest time, I've been wanting to create a simple feed widget to place on a web page and have it display links from the syndicated site. Easier dreamt than done, until this thanksgiving weekend when I decided to partially put the long holiday to good use and write up a widget tool. It's a weird thing. I get to be away form work for 4 days, yet I spend my free time coding and learning new programming techniques. There was certainly time spent with family, but for me, delving into coding challenges also has a therapeutic effect. Some programmers may understand the absurdity of this. There's that drudgery factor at work that you don't have to deal with when hobby programming. No boss to report to, no business politics, no users interrupting with questions or problems. I get to set the rules from end to end. Not to mention that at work I'm an ASP.NET programmer, and at home I turn into a PHP coder. I realize that there are a bunch of these RSS/ATOM widgets already out there. So mine's another one to add to the list. Partial as I may be, I think it does a decent job and it allow for some basic customizations. Since this is a new utility, consider it in beta mode until I say it's not. If Gmail can be in beta for eternity, why not this one? I've tested it quite a bit, but I'm sure there are still bugs to be fixed. Also I have no idea what impact this may have on my Web hosting account quotas. We'll find out. Documentation is not one of my strengths, but I will eventually produce an FAQ on some of the widget's capabilities, features, and customizations using, for instance, CSS. There are also plans to enhance it further, if there's enough demand for it. For now, feel free to visit the RSS/ATOM widget tool page, create your JavaScript RSS/ATOM widget, and embed it in your Web pages. Let me know if you run into any bugs or have any suggestions. I'll try to address them as time permits. rss,atom,rss widget,xml,web 2.0,ajax,javascript Labels: javascript, web, xml <RSS ATOM Widget Utility> // posted by rh 1 comments Saturday, November 17, 2007 Adobe Flash Cookies, Local Storage  Call me paranoid, but I always like to be clear on what a site is doing on my computer while I have it up on my browser. I don't think I'm that paranoid (okay, maybe a little,) but I think users are entitled to know if a site is storing and reading data to and from their computers. That's an exact description of what browser cookies are used for, but tonight I learned about a new kind of cookie I had been abashedly unaware of. I for one, don't appreciate being followed around while I surf the Net, so I delight in cleaning or modifying my cookies often and throwing off the trackers. But tonight I was stumped by an often visited music site that remembered me steadfastly, even when I deleted my cookies and visited the sites on different browsers. What was going on here? Was the site using my IP address to identify me? I doubted it. IP address tracking is so inexact these days that it's generally only used for geo-location, and even that yields questionable results. After some persistent searching, I finally found the elusive answer, and it was a Flash cookie. I had never known that Flash player could store data on users' computers, so I delved a bit deeper. I found out that they are similar to, but work completely independently from browser cookies. On my Windows XP machine, Flash cookies (known as local storage in Macromedia's jargon) are stored under the "\Documents and Settings\[account]\Application Data\Macromedia\Flash Player\#SharedObjects\" folder. By default each site is allowed to store and access up to 100KB in the cookies and users are oblivious to this activity the whole time. Way to go Adobe. I don't remember every seeing anything about these cookies when I was installing the Flash player. No doubt it was buried in some privacy legalese. The good news is that once discovered, these cookies can be deleted using Adobe's Website, or by just simply zapping them in the folder mentioned above. Adobe's site also allows users to disable Flash cookies altogether. The caveat is that just like browser cookies, many sites rely on these cookies and probably will not function correctly without them. For me, the happy medium was to configure Flash to alert me every time a site wants to store a Flash cookie on my PC. You can do the same by going to Global Storage Settings panel on Adobe's Flash site and alter your settings to match the figure below. Maybe you can't stop sites from storing and retrieving data from your computer, but at least you know who's tracking you using this type of cookie. flash cookies,flash local storage,adobe,macromedia,flash,cookies,browser cookies,browsers,flash player Labels: browsers, flash, web <Adobe Flash Cookies, Local Storage> // posted by rh 1 comments Tuesday, November 13, 2007 Web Hosting Customer Service  Here's the brilliant idea for today. Why don't the hosting companies let the right representatives answer their customers' questions, instead of letting customers waste time with their less trained agents? I've been helping a friend setup a site on one of the nationally known web hosting sites, and I must say getting the right answer is an exercise in extreme patience. My own site (the one carrying this blog) is hosted with one of the smaller outfits and I've been able to get my technical questions promptly and correctly answered most of the time. That doesn't seem to be the case with the larger companies. I don't even think they read the emails. I ask for something to be disabled, they send instructions back on how to enable it. Even worse, after dealing with so many newbies mindlessly spouting techie jargon, they have developed resistance to anyone trying to get technical with them. It's only after several tries that an issue is escalated to the more knowledgeable staff who can finally help resolve the issue. I'm not sure whom is to blame here. Web hosting must be a cut-throat business and these outfits are just holding the line on expenses. The front-end positions are generally low-paying jobs, so I assume new people are brought in, trained quickly on the web hosting's interface, and then dumped in the call center pool. I wish I could get a sample of the questions these guys field. I can only imagine what absurdities customers throw at them. The front-end agents are probably fine for the amateur callers, but it’s the back-end guys I really need when I call or email for something. I presume I never quite make it to the back, but I would be quite content with the middle-layer without being hassled by the front guys a few times. People tend to note that smaller hosting companies have more informed support personnel. That's certainly true with my hosting service. Perhaps the bigger guys can learn something from their smaller counterparts. web hosting,tech support,call centers,customer service Labels: customer service, web <Web Hosting Customer Service> // posted by rh 2 comments Tuesday, October 23, 2007 Proxy Server Woes  Troubleshooting misbehaving Web applications is a part of my job at work. Most of the time it's some buggy code that I or someone in our team had written. But sometimes I get approached about system problems too, like when a browser is having connection issues and can't access Web sites. If you are in the help desk business you know how the question is posed, "Is the Internet down?". I've always been tempted to reply: "Why yes, it is down. All that redundancy worked into it back in the DARPA era was a pile of lies. The whole Internet is toast." Most of the time browser problems are not browser problems at all, they are network issues. The browser just gets the blame because it's front and center. A bad cable kicked once too many, an unplugged Wi-Fi access point, or a beleaguered DHCP server are the usual suspects. But the one that trips me most of the time is the proxy server setting. I've gotten better at it but I still occasionally miss this little item from my checklist. As you may know proxy servers are protocol-level servers (appliances) that work as brokers between clients and servers. The most commonly used is the Web (HTTP) proxy server. Traffic passing back and forth between browsers and servers is funneled through them. Why use them? A detailed discussion is too boring, but in a nutshell, they make network admins' lives easier (more granular control, central management, etc), and they can cache and compress data saving on bandwidth and thus speeding things along. Proxy servers are helpful, as long as one remembers that they're being used, especially when troubleshooting. Case in point, the other day I was working on a laptop connected to the Internet over Verizon Wireless broadband EVDO. No matter what I did I couldn't browse to the laptop's local Web server. Everything was fine when wireless was off, but as soon as I started it up, the local Web server would become inaccessible. What was happening? I could successfully ping my own server (duh), the Web service was running fine and listening on the local address, and there was no proxy server setting hidden in my browser (obvious place to look, but easily missed, specially when frantic). After hours of investigating, including recruiting assistance from a colleague, the issue was revealed, quite by accident. Verizon Wireless comes with an add-on product called Venturi (enabled by default), used for optimization and compression. Turns out Venturi was the cause of this issue. When disabled, everything began working as expected. Venturi inserts itself in the TCP/IP stack and reroutes Web requests to a proxy server. No wonder I couldn't browse to the local server, the external Venturi proxy server had no idea how to access it, nor anything else on the private LAN. The trouble with proxy servers is that it's hard to detect that your applications are using one, unless you pay attention to the telltale signs. First, with most applications, including browsers, you can check the network options and see if a proxy server address is configured. Next you can visit an IP address and header lookup page like whoami/My IP Address on this site, and check out the results. This page and the likes will display whatever information they can glean from your connection. Sometimes, but not always, proxy servers add headers such as "X-Forwarded-For" or "Via" to reveal the actual source of the request. You can also compare the shown source IP address to the IP address of your own machine. From the command line enter the "ipconfig" command (in Windows) which will display your actual IP address. If the lookup page is displaying the same IP address, chances are there is no proxy server in between, otherwise your traffic may be going through a proxy server and that would explain the irregularities of the sorts I was experiencing. proxy servers,verizon wireless,evdo,ip address,ipconfig,tcp/ip,networks,browsers,http Labels: networks, technology, web <Proxy Server Woes> // posted by rh 0 comments Monday, September 03, 2007 Google's GrandCentral  The other day I received an email from GrandCentral with the subject line: Invitation to sign up for GrandCentral. Like everyone else I receive plenty of bogus invitations to sign up for this or register for that. It smelled like spam. "Who the hell is GrandCentral?", I thought. But before clicking the spam button, I decided to check the email. It started: Good news! We are excited to announce that we are opening the GrandCentral private beta to some additional users and would like to extend you an invitation to sign up. Then I realized that back in June when the news of Google purchasing GrandCentral had hit the wires, I had added myself to their waiting list. So I proceeded with the registration and got my own number. This is a pretty neat concept. You get to choose a phone number and you can link it to several physical numbers. Then, depending on your choice, an incoming call will ring all or some or none of the numbers. It comes with voicemail, Caller ID, email forwarding and a number of other features. Most activities can be done online as well as over the phone. It's presence, call forwarding, and messaging all wrapped in one package. Best of all it's free, and you can keep the number for life. As things are with Google products, GrandCentral is in Beta and probably will be for years. It remains to be seen how Google will fully monetize GrandCentral. There are some paid features, I believe, but I assume the majority of users, myself included, will only use the free services. So I presume, like most Google properties, GrandCentral will come to rely on advertising for a big portion of its revenues. That will probably include text ads on the site, voice ads inserted before or after playing voicemails, or ad links included in notification emails. I wonder if GrandCentral future plans include free fax service. That would be a natural progression and it would position them against some long-established services like jConnect that offer free fax numbers and whom I have been a satisfied user for a number of years. grandcentral,google,fax,phone service,fax service,presence,voicemail,grand central,call forwarding,messaging Labels: google, technology, web <Google's GrandCentral> // posted by rh 2 comments Wednesday, August 08, 2007 Web Scratchpad, Notepad, Clipboard - PADFLY.COM  The other day I was chasing a problem on one of our servers. While searching for a solution on my desktop I happened upon a web page that seemed to present a solution. At this point I wanted to go over to the server, pull up the web page and try some of the suggestions. There was only one problem. The URL of that page was one of those long ones with a number of numbered parameters. My choices were: Open Remote Desktop to the server from my PC, fire up the browser on the server, copy and paste the URL into the browser and continue working via Remote Desktop. That's fine and good, except that sometimes copying and pasting from my local PC to Remote Desktop doesn't work. Besides If I wanted to try this on several other servers later on, this solution would be impractical. Commit the URL to memory and then try to recall it at the server's console. Yeah right. Everyone knows how flawed a programmer's memory can be, specially with a long URL that you have to get it exactly right. Copy the URL to Notepad, save it on a shared folder on my PC and access it from the server. Email it to myself and then access it on the server over the web. That would have meant a number of steps including logging in to my account from the server. Use a service like Yahoo Briefcase or Google Notepad and access them from the server, but again there was that login step again and I'd have to remember to log off, lest someone else might get access to my private account. In short, a simple task of saving some text and retrieving it from elsewhere is a cumbersome chore. That's when I stumbled upon padfly.com. This web-based service allows one to copy some text to an easy-to-recall URL (like http://padfly.com/MySavedURL) and then access it from anywhere at any time. padfly is free, requires no login, and the saved content can be accessed and edited at any time, by anyone, from anywhere. You can copy any piece of text to a path name of your choice (called a PAD) and then have anyone on the other side of the company or the other side of the world view it. I actually used padfly today and it definitely delivered for me. So the next time you need to quickly save and recall a piece of text or share it with others, give padfly.com a try. It might save you a lot of time and hassle. It's a simple web-based scratchpad, notepad, and clipboard you might have been searching for. There is a footnote to this blog entry. padfly was actually developed by yours truly last weekend to address the issue I mentioned up above. It was handy enough for me that I decided to tidy it up and put it online. Send me a note if you have suggestions for features, improvements, or other ideas for padfly. Enjoy. scratchpad,clipboard,online notepad,web-based notepadcollaboration,url Labels: web <Web Scratchpad, Notepad, Clipboard - PADFLY.COM> // posted by rh 0 comments Sunday, July 15, 2007 Homepage Hazards  Sites, specially news-related or educational, usually cram their homepages with links to various sections and freshly updated pages. In that regards those homepages are portals into the rest of their respective sites where the real content resides. That's all fine and good until they display links from those sections that the site maintains little control over. Forums, for example, are one these notorious areas trolled by spammers and jokers. The problem is that by nature they are supposed to be democratic. Pre-moderated forums generally suffer from anemic posts and little lively action. On the flip side, unmoderated or post-moderated forums spur real-time discussions, but invite nuisance posts. This is depicted in the image grab from the homepage of devx.com, a development site frequented by programmers and, in this instance, linking to a prankster's or a spammer's post in one of their forums. The offending post was removed at some point, but the orphaned link remained on the homepage until it was pushed out by newer links. homepages,forums,spam posts Labels: spam, web <Homepage Hazards> // posted by rh 0 comments Wednesday, June 13, 2007 Apple Safari on Windows  I'm a man of habits. That trait also extends to my browser of choice and, like many, I use Internet Explorer (IE) to surf the Web. Years ago, when the Internet was still new to the general public, it took me some time to actually start using a browser alongside my favorite text-based programs to browse the Web, read Usenet, or check email. That was the NCSA Mosaic times. Then I took my time to switch to Netscape. And I was yet again behind the curve when Microsoft joined the fray and introduced IE. For now I'm still an IE user, and true to form, I have refused to upgrade to version 7. Not that all the bad publicity has helped anyways. I did try Opera once and saw no need for it after fiddling with it for a few days. I do use Firefox occasionally now. Not because I like it any better than IE for general browsing, but mostly to test Web pages. Firefox does have a leg up on IE in one area, the add-ons. Unlike IE, Firefox has done a superb job in designing and integrating the add-ons. They are much more straightforward to program and there's a bevy of available add-ons on the Web to choose from. Greasemonkey is one of my favorites, for example. One browser I wished I could have was Apple's safari. That is the browser of choice for most Apple junkies, but until now it was out of the Windows' realm - Until now. Apple finally released a version (public beta 3) and, true to their claims, it is faster than either IE or Firefox. The speed was even evident during the installation process. It has a relatively small installation file and the setup process was fast. Safari for Windows is a no-frills browser. It's lightweight and doesn't have a lot of bells and whistles. It does have an Apple-ish look, but after browsing to several sites I could confirm that their claim of being speedy is for real. I didn't clock its rendering speed, but it did feel faster than its other two popular counterparts. The configuration is a bit clunky and bears some resemblance to Firefox (must be the Mozilla heritage) and the fonts are a bit rough, but it performs magnificently. I was impressed. Having Safari available on Windows is also a boon to site designers who need to check their pages for browser compatibility. One of the pages I tested was the JavaScript Countdown page on this site. There were some complaints of Safari incompatibility from some of the users. I was prepared to see a broken counter, but to my surprise the page loaded just fine, counter and all. That left me wondering whether there were differences between the Windows and Apple versions. Since I don't have access to an Apple, I am going to tentatively declare that the countdown utility is Safari compatible. Safari won't unseat IE for me as the browser of choice for general browsing, but if you are in the market for a lightweight and speedy browser, this could be the one. But even if not, it's a valuable tool for Web designers toiling on Windows to achieve maximum browser compatibility for their pages. If you want to download the browser, here's the link: Safari for Windows. safari,apple,apple safari,ie,internet explorer,firefox,mosaic Labels: browsers, web <Apple Safari on Windows> // posted by rh 0 comments Thursday, June 07, 2007 HTTP Authorization and .NET WebRequest, WebClient Classes  One of the more useful, yet simple, classes of .NET is System.Net.WebClient. It basically simulates a simple browser to interact with web pages within your code. You can use it to GET pages or POST data to pages over http or https (SSL). The page response can be saved into a byte array, a string or a file. WebClient can even be used to pass authentication values to pages that require it, you know, those pages that pop up a user and password window before letting you in. For that, you’d supply the values to the Credentials property of WebClient and fire away. Recently I was trying to access a protected page using WebClient. The code was pretty straightforward: WebClient wc=new WebClient(); wc.Credentials=new NetworkCredentials("user","pass"); string a=ws.DownloadString("http://www.example.com"); Pretty simple, eh? I've done this a million times, but in this case (the actual site shall remain nameless) it was throwing an exception. After some investigation, I noticed that the page was returning a 404 code (page not found) prompting the exception error. I also discovered that the call wasn't sending the appropriate Authorization header to the page. The site's documentation was clear about accessing the page using basic authorization header, but there was no getting past the exception. What was going on here? After some fruitless troubleshooting, I decided to forego the Credentials property and manually craft the Authorization header. To do that I wrote the following code: WebClient wc=new WebClient(); wc.Headers.Add("Authorization","Basic "+   Convert.ToBase64String(   Encoding.ASCII.GetBytes("user:pass"))); string a=ws.DownloadString("http://www.example.com"); This time, to my delight, the page obliged and the string variable "a" received the page's content. Not being satisfied with merely solving the problem using a different route, I decided to dig in and find out why the original (more proper) code was failing. First I discovered that the page was designed to return a 404 code rather than the customary 401 (Not Authorized) when the credentials were missing. I'm not sure what the RFC's position on this is, but according to MSDN documentation, when a protected URL receives no authorization header from a client, it should return a 401 code, signaling to the client that authentication is required. The client should then provide the authorization header with each access, satisfying the URL's demand. The WebClient class with its Credentials property is designed to do just that, but not in a straightforward manner. Under the hood, WebClient constructs a HtttpWebRequest object and sends a plain request to the specified page. Upon receiving a 401 code, it crafts the authorization header using the Credentials property and hits the page again. That's two round trips for every request. Worse yet, it does that for every subsequent request. I fail to see why WebClient insists on not sending the authorization header in the first place. After all, if the coder specifies the Credentials property, he must already know that the page requires authorization and WebClient should just obey and send in the header without the fuss. In this case, the site's response aggravated matters by sending back a 404 (rather than a 401) after the first request, sending the whole process into a tailspin and causing an exception to be raised. If you use the WebClient class in your .NET code to access protected URLs, watch out for this little stumper. It wasted quite a bit of my time. Maybe my loss will be your gain. .NET,dotnet,webclient,webrequest,httpwebrequest,authorization header,404 code,401 code,http codes Labels: programming, web <HTTP Authorization and .NET WebRequest, WebClient Classes> // posted by rh 1 comments Saturday, May 19, 2007 Yahoo Messenger  I guess this means I'm old and not with the current times, but I just don't get the lure of instant messaging (IM). The whole experience is so unsettling that a long time ago I vowed not to ever use it. What's the point? I have to be a super-typist to have a barely coherent conversation. Then there are all those typo-ridden acronyms (lol, rotfl, u2, b4, cu, wtf) to learn. And if there are more than two people chatting, forget about keeping track of who said what, responding to whom and to which comment. Who needs the hassle. If I need to send a message, I just use the good old-fashioned email. And for urgent cases, I just pick up the handset. I remember back when IRC chat rooms where in vogue. Out of curiosity, I peeked inside one of the rooms to see what the big deal was. After about two minutes of observing the volley of nonsense by purported teenagers, I had enough. That was my first and last time in an IRC chat room. So I didn't think I would ever use a messenger program, but surprisingly I have been a happy user of Yahoo Messenger for a couple of years now. No, I don’t use it for IM, voice, or webcam. But I do use it to listen to music, read RSS subscriptions, and check a list of stocks. Yahoo Messenger has become a useful tool for me to that end. Anyways, a few days ago I received this popup that alarmed me for a moment. I thought I had picked up a spyware or an adware. Turns out that it was just a system message from Yahoo Messenger. Right, I'm so glad that I can now save my chat logs on Yahoo's servers. Guess I won't be exceeding my storage quota any time soon. messenger,yahoo messenger,irc,irc chat,im,instant messaging Labels: im, instant messaging, irc chat, technology, web <Yahoo Messenger> // posted by rh 2 comments Wednesday, May 09, 2007 End of the Road for Old Blogger  When Google bought blogger.com from Pyra Labs over 4 years ago, there was a lot of head-scratching. Blogosphere wasn't new but it was a fraction of what it is today. Google wasn't new either, but it was just search engine site, a really good one. And it still was just a private company. Today, blogger.com is just a part of Google's public empire and it has become a popular site for the myriad blogging enthusiasts. Count me in as one of the enthusiasts. I have been a pretty satisfied user of blogger.com since I started blogging on this site. About 2 years ago Google set out to revamp blogger.com by adding new features, improving some of the existing ones and, most prominently, migrating the accounts to its own login infrastructure. I converted this blog to the new version a few months ago. The process was relatively smooth with a few minor glitches. Today blogger.com finally retired its old version for good, hailing: "Old Blogger is dead! Long live Blogger!". What tipped me off was the missing blogger image button at the bottom of my blog pages. According to blogger.com's terms of service users are (or were) required to display this button on their pages. I am not sure if the terms of service still require displaying this button. Right now it seems flaky (look to the right). Sometimes it loads, other times it fails. For now I'm going to leave it, but if it continues to remain missing, I'll assume that it's now a relic of the past and its presence is no longer required. Then perhaps it's time to toss it. google,blogs,blogger,blogger.com,pyra labs Labels: google, web <End of the Road for Old Blogger> // posted by rh 0 comments Thursday, May 03, 2007 Microsoft's bCentral LinkExchange Banner Network Shuts Down  I Received the inevitable email from Microsoft today. It was inevitable because in the face of all the acquisitions, consolidations, and new technologies to deliver ads on the web, it was a miracle that LinkExchange even lasted as long as it did. LinkExchange opened its operations in 1996. It created a banner exchange marketplace where sites could get their banner ads displayed on other members' sites in exchange for participating in the program and displaying banners from others. The company made money by selling a percentage of the banner placements to paid advertisers. In 1998 (fortuitously before the dotcom implosion) Microsoft acquired LinkExchange for $265 million and rolled it into its small business services initiative, dubbed bCentral. Eventually newer players (read Google) and newer technologies made the old boring banner exchanges obsolete but LinkExchange soldiered on, until now. Now that Microsoft is shifting its bCentral operations to live.com and adCenter has been positioned to compete with google's AdWords and Yahoo's Panama, it was time to decommission the old banner exchange. Microsoft stopped taking new LinkExchange applications on Nov. 15th, 2006 and as of June 4th, 2007 will stop serving banners. So as LinkExchange takes its final bow, scroll to the bottom of this page to say your farewells. Soon there will be an empty spot in its place. Microsoft,LinkExchange,banners,advertising,bCentral,banner network Labels: advertising, Microsoft, web <Microsoft's bCentral LinkExchange Banner Network Shuts Down> // posted by rh 0 comments Tuesday, February 13, 2007 The Alternate Chat Room  On September 4th 2006 the world lost a famed and intrepid animal lover, Steve Irwin. Soon after, the Internet was abuzz with the news of his tragic death. Checking the news on the next day, during my lunch hour, I scanned some of the stories surrounding the circumstances of his death and this is one of the pages on MSNBC I stumbled on covering the story: http://www.msnbc.msn.com/id/14663786. I then proceeded to read some of the comments left by the viewers. As expected, most contained messages from bereft fans. There were also a number of messages from readers commenting on his carelessness with animals or the fact that he got what he deserved as well as a number of irrelevant messages from jokesters. I thought that everyone was entitled to their opinion and I moved on. A couple of months later, as the year was about to turn over, I happened to revisit the same story linked from a page on MSNBC touting the most popular stories of 2006 and I was astounded by the number of comments left on that page. As I recall, some 70,000 entries had been logged by the end of 2006. I was in disbelief over the sheer number of readers that had left comments on that page and I decided to read some of the latter ones. As I read some of the comments a feeling of confusion set in. The messages had no relevance to Steve Irwin or his legacy. They weren't even about animals or nature. They seemed like random ramblings posted by a few readers, but as I read a few more a pattern started to emerge. The message board was actually being used by individuals to pass messages to each other. Real time conversations were taking place as I was scanning the board. Topics ranged from dinner preparations, to old friends, to buying a new car with a number of people typing one-liners back and forth. This board was no longer a comments section about Steve Irwin but it had morphed into a private chat room for a few people engaged in private conversations, a substitute for a chat room or Instant Messaging, if you will. Only the messages weren't so private as they were out there for the world to read. As of this writing that board has logged over 100,000 messages. New posts continue to pour in and I'm willing to bet that majority of them have nothing to do with the famous crocodile hunter. Apparently the admins at MSNBC have no problem with this. Comments are most likely scanned by a program to weed out obvious profanity and people are left to their own devices to manage the board. That level of freedom coupled with endless computing resources (storage and bandwidth) at MSNBC opened the door to an alternate usage of the board many took advantage of quickly and in force. This just goes to show how unpredictable and ingenious the online community really can be. Many forum and chat sites would dream of having the level of traffic and interaction that this one obscure board has attained and surely there are many more boards of this type living in the shadows, unnoticed by the casual browser or even the site administrators. The fact that such usage is improper is besides the point. What is notable is the level of people's inventiveness to bend and tailor a simple and trivial product into a makeshift, yet effective, tool for communication. There's a business lesson in there waiting to be discovered. chat rooms,instant messaging,im,steve irwin,crocodile hunter,msnbc Labels: web <The Alternate Chat Room> // posted by rh 1 comments Wednesday, January 10, 2007 HTML Forms, Part 2  Click here to read Part 1 There are really many steps that a browser goes through to display a web page or post some data to it. For the purposes of this discussion, a simplified process consists of a request and a response. The browser makes a request to a server for a particular page, and the server returns the data to the browser in the form of a response. There are a few types of requests a browser can issue using commands that are generally known as 'verbs'. The most widely used verb is GET. Whenever you visit a web page, your browser issues a GET command to the server along with some other parameters and the server obliges by returning the data for the particular page. In fact you are reading this very page because your browser made a GET request to www.hashemian.com and specified this page. The server then returned this page to your browser in a raw format. Your browser then rendered it as you see now. But what happens when you submit a form to a page? In that case the page generally contains a FORM tag like this: <form method="POST" action="some_page"> some form elements such as <INPUT> here a submit <BUTTON> here </form> Notice the POST verb specified as an attribute of the form tag. When you click on the Submit button, the data you entered on the form is sent to some_page using the POST verb. The POST verb is similar to the GET verb but there are some additional data. The browser contacts some_page, issues the POST command, it also specifies Content-Length and Content-Type. Content-Length is the character length of the data that you have submitted to the page, while Content-Type specifies the format of the data. The data format, known as MIME type, is generally specified as application/x-www-form-urlencoded. That signals to the server that data was encoded in a specific format so the server can figure out how to decrypt and use the data. We'll delve into more details on POST in part 3. html,forms,get,post,url Labels: programming, web <HTML Forms, Part 2> // posted by rh 0 comments Tuesday, December 26, 2006 HTML Forms, Part 1  My first experience with the World Wide Web about 13 years ago was in school using NCSA Mosaic running on Sun SPARCstation hosts. Mosaic was the original graphical Web browser from which all modern Web browsers today can claim their common ancestry. It was a giant leap from text-based applications such as Usenet readers that were much more common back in those days. Soon after Tim Berners-Lee invented HTTP, a bunch of developers from the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign got together and Mosaic was developed. The resulting work was eventually licensed to Netscape and Microsoft. The Netscape and Internet Explorer browsers were born out of that licensing and the rest is history. While today's browsers are much more versatile and dynamic, they are essentially the same utilities at their core as the original Mosaic was. The user initiates a request to a Web server, and a response is returned and displayed on the browser. Initially the interaction with the Web servers were one-sided and static in nature. A page was requested and data consisting of text, images, and hyperlinks were returned to the browser. Tags were used to position and style the elements on the browser area. But soon all that changed when web pages were armed with forms from which users would be able to send data back to the servers. Of course browsers can’t just sent any arbitrary data to the server. The type and format of that data is dictated by the browser. Data not conforming to what the server expects will be, at best, ignored. But what is the mechanism by which data is sent to a server? For that we need to study the two most common request types (a.k.a. verbs) browsers send to servers; GET and POST. We'll look at these two request types in part 2. ncsa,web browsers,www,netscape,http,web servers,html forms Labels: browsers, web, web servers <HTML Forms, Part 1> // posted by rh 0 comments

Links Technorati Profile TMCnet.com ARCHIVES 09/01/2003 - 10/01/2003 03/01/2004 - 04/01/2004 04/01/2004 - 05/01/2004 05/01/2004 - 06/01/2004 06/01/2004 - 07/01/2004 07/01/2004 - 08/01/2004 08/01/2004 - 09/01/2004 09/01/2004 - 10/01/2004 10/01/2004 - 11/01/2004 11/01/2004 - 12/01/2004 12/01/2004 - 01/01/2005 01/01/2005 - 02/01/2005 02/01/2005 - 03/01/2005 03/01/2005 - 04/01/2005 04/01/2005 - 05/01/2005 05/01/2005 - 06/01/2005 06/01/2005 - 07/01/2005 07/01/2005 - 08/01/2005 08/01/2005 - 09/01/2005 09/01/2005 - 10/01/2005 10/01/2005 - 11/01/2005 11/01/2005 - 12/01/2005 12/01/2005 - 01/01/2006 01/01/2006 - 02/01/2006 02/01/2006 - 03/01/2006 03/01/2006 - 04/01/2006 04/01/2006 - 05/01/2006 05/01/2006 - 06/01/2006 06/01/2006 - 07/01/2006 07/01/2006 - 08/01/2006 08/01/2006 - 09/01/2006 09/01/2006 - 10/01/2006 10/01/2006 - 11/01/2006 11/01/2006 - 12/01/2006 12/01/2006 - 01/01/2007