Hashemian Blog

Web Tools, Financial Markets, Technology

Sunday, November 23, 2008

News Robot Incompetence

I am a fan of Google Finance. There's a lot to like. Real-time stock quotes, nice charts and graphs and a clean design. But when it comes to breaking financial news, Google just doesn't cut it. Many sites use bots to collect and disseminate news, but in the rapidly changing financial world a news bot is all but useless.

Take a look at the snapshot below taken last Thursday from Google Finance's homepage. The image has been tightened up to show the interesting elements. The headline and subordinate stories shouted losses in the stock market, while the real-time numbers and chart showed considerable market gains.

This contradiction was brought to you courtesy of Google's news bots. By the time the headlines came online, the stories were no longer relevant. Of course, the stock market (specially in volatile times) is a rapidly moving target, making it difficult for an even experienced editor to keep up, let alone a soulless bot who can't care less about market timeliness, or which stories may matter more to the readers.

So for now, Google Finance is fine for stock quotes and charts, but for timely news its bots have a long way to go to catch up with the biological machines known as the editorial staff.

google,news,google finance,market data,stock market
Labels: financial, web
<News Robot Incompetence>

// posted by rh

0 comments

Thursday, November 13, 2008

ASP.NET Response.TransmitFile File Locking

If you use the Response.TransmitFile method in your ASP.NET pages, there is a good chance that the file being transmitted is locked and unavailable for writing during that operation. TransmitFile was introduced with .NET V2.0 and it's supposed to be a more optimized version of WriteFile.

For months I had been dogged with exception alerts when a scheduled program was trying to update files used in TransmitFile. As the alerts were intermittent and the files would eventually get updated by the program, I would just ignore the messages. But today I decided to take a closer look and discovered that TransmitFile is to blame for the files being locked.

As I understand, TransmitFile in ASP.NET makes use of the Windows TransmitFile API function which directly streams a file to an open socket. It is more optimized than WriteFile which buffers the entire content of the file before emitting it. In that sense, WriteFile consumes precious memory and processing resources, switching between kernel and user modes, to accomplish its work.

While TransmitFile API function is optimized, care must be taken to avoid having the streamed data getting mixed with other data and my hunch is that some locking mechanism is used to assure an orderly and sequential transmission of data under ASP.NET. That's all fine until one needs to write to the file which could get shot down while the lock is still active. I was able to reproduce this behavior in a loop where a file is continuously updated and then streamed via TransmitFile. Here’s a simple example:

void Page_Load() {
  Response.Buffer=false;
  for(;;) {
    System.IO.File.WriteAllText(MapPath("/test.txt"), DateTime.Now.ToString());
    System.Threading.Thread.Sleep(100);
    Response.TransmitFile(MapPath("/test.txt"));
    Response.Write("<br>");
  }
}

Running this ASP.NET page invariably lead to an exception being thrown. I then changed the code as shown below:

void Page_Load() {
  Response.Buffer=false;
  for(;;) {
    System.IO.File.WriteAllText(MapPath("/test.txt"), DateTime.Now.ToString());
    System.Threading.Thread.Sleep(100);
    // CHANGED LINE BELOW
    Response.Write(System.IO.File.ReadAllText(MapPath("/test.txt"))); 
    Response.Write("<br>");
  }
}

I ran the page a number of times and no exception was thrown. I'm sure that ReadAllText is nowhere as optimized as TransmitFile, but I would readily take the performance penalty over the unpredictable exceptions. Incidentally, WriteFile proved to be no better than TransmitFile in locking the file.

Based on the simple test I have concluded that TransmitFile or WriteFile should be used when the file is never or rarely updated. If the file is to be updated frequently, ReadAllText (or other System.IO file reading operations) is the way to go. But if someone can disprove my tenuous theory, by all means.

asp.net,file systems,windows,web programming

Labels: programming, web

<ASP.NET Response.TransmitFile File Locking>

// posted by rh

0 comments

Wednesday, November 12, 2008

MSNBC Prostitution Articles

For the past few days a number of prostitution-related articles have made their way to the "Most Viewed" list on MSNBC, but clicking on those links results in a Page-Not-Found error. Is this a prank by vandals or just MSNBC's censors blocking the articles that were ostensibly published on the site?

Screenshots:

msnbc,prostitution,censorship
Labels: censorship, web
<MSNBC Prostitution Articles>

// posted by rh

0 comments

Tuesday, November 04, 2008

Why I Voted Obama Today

I am an independent in just about all facets of my life, including in my political choices. It's a simple belief that not one party is right or wrong all the time and either side can produce good or bad candidates.

While many have supported or opposed Obama because of his economic policies, my vote went to the person that would be less likely to start a war. Having subsidized (through tax payments) one unjust war, my conscience could not bear the responsibility of financing another. Under Obama I may pay higher taxes, I may subsidize the poor, or I may pay more to heat my house, but for me the stakes were much higher. The risk of contributing to the demise of more innocent lives was much greater under a McCain administration.

The Iraq war was based on fabricated evidence. We went to Afghanistan to capture a terrorist mastermind, but somewhere along the way we were railroaded into another war that was anything but rightfully justified. The terrorists who committed the 9/11 atrocities were Saudi Arabians, financed by a Saudi. They were trained in Afghanistan. Most transited through Europe and they got their flight lessons in the US. There was no connection to Iraq and no weapons of mass destruction were ever found there.

In the ensuing years American forces and Iraqi citizens have sustained heavy losses. For what? Bin Laden and most of his cronies are still at large, while the architects of the Iraq war (Bush, Cheney, Rumsfeld, Wolfowitz) gloated over the capture and hanging of Saddam Hussein. Yes, he was a brutal dictator but also a one time pal, armed to the teeth by the same powers that eliminated him.

Reminds me of the song "Fortunate Son" by Creedence Clearwater Revival:
Some folks inherit star spangled eyes,
Ooh, they send you down to war, lord,
And when you ask them, how much should we give?
Ooh, they only answer more! more! more!
Well, today I decided not to give them any more!

war,iraq,afghanistan,bush,obama,mccain,elections
Labels: politics
<Why I Voted Obama Today>

// posted by rh

0 comments

Monday, November 03, 2008

ASP.NET Denial of Service

How to send your IIS server into a frenzy with one line of HTML code? I didn't think it was possible until a few days ago we were stung with a persistent denial of service at work. This is what the event log showed on every instance of outage:

Event Type: Warning
Event Source: W3SVC-WP
Event Category: None
Event ID: 2262
Date:  10/23/2008
Time:  8:32:51 PM
User:  N/A
Computer: WEB
Description:
ISAPI 
'C:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727\aspnet_isapi.dll' 
reported itself as unhealthy for the following reason: 
'Deadlock detected'.

I have always been frustrated with IIS event logs and there's one proof of that. OK, I know there was a problem, but can this be more specific? which process? which application pool? Which page? This is hardly helpful. I believe new Failed Request Tracing feature in IIS7 was designed to help with just that. Countless debugging hours would be saved if one could quickly identify a misbehaving page. Googling this entry lead me to the http://support.microsoft.com/kb/821268 KB article and that sent me on a wrong path for some time before realizing that I was on a fruitless chase.

Finally, After 6 hours of struggling with the server (Window Server 2003, IIS6, CLR 2, FCL 3.5) and slicing and dicing and moving various applications to different pools, I found the offending ASPX pages. It didn't take long for my colleagues to discover an HTML anomaly in those files. And here it is in a generic format:

<p param1="" param2="" param3="" param4="" param5="" 
param6="" param7="" param8="" param9="" param10="" param11="" 
param12="" param13="" param14="" param15="" param16="" param17="" 
param18="" param19="" param20="" param21="" param22="" param23="" 
param24="" param25.="" />

Notice that last parameter with a trailing period (.)? that's the culprit right there, an HTML tag with a long list of attributes and a punctuation mark in one or more of the ending parameters. It had crept into our pages via a poorly made web editing product.

To test for yourself, just drop the line into an ASPX file and browse to the page and watch the CPU crank up and eventually the server refuse to serve pages. Remove the period or reduce the parameters and the page will display fine. I know, seems hard to believe. I didn't believe it either at first. But the results were the same on IIS6 and IIS7 on various platforms.

My hunch is that this tag throws the ASP.NET engine into a regular expressions frenzy as it tries to construct the page elements and compile the page into a DLL application. Now this HTML syntax might look weird but I don't think it's illegal and it certainly shouldn't cause a denial of service. Certainly IIS or browsers have no issue with HTML files containing such tags.

While at first I had feared serious security risks, this issue has a limited risk factor. In order for an attacker to exploit this issue, he would need access to the page sources. It's not something that can be injected in or remotely scripted into the page. Of course someone with a shared hosting service, could potentially take down the entire server and all the sites along with it.

So if you run into such a problem on your IIS web site, you might want to check your source files for these types of HTML tags. And by the way, Microsoft has already been advised of this issue and they have indicated a fix will be incorporated in the next ASP.NET release.

asp.net,denial of service,aspx,dos,ddos,iis,html,microsoft

Labels: Microsoft, web

<ASP.NET Denial of Service>

// posted by rh