Hashemian Blog
Web, Finance, Technology, Running

January 11, 2019

Dabbling in IPv6

by robert hashemian @ 4:02 pm
Filed under: internet — Tags:

Nowadays many people know of IP addresses. Those 4 numbers (technically known as octets) separated by dots (e.g. 173.162.146.61) which connect all of our devices to the Internet and allow them to find each other. Of course many don't understand the underlying technologies that make the whole thing work and they don't really need to understand it.

That IP address that most people know is IPv4 and is almost 40 years old. Its address space was depleted back in 2011, as in there were no more addresses for the overseeing organization, ICANN, to hand out.

Back in 1999 realizing the dire situation of IPv4, Internet Engineering Task Force (IETF) developed a new IP scheme known as IPv6. While IPv4 is 32 bits allowing for about 4.3 billion addresses, IPv6 is 128 bits allowing for 3.4×1038 addresses. It’s a big number, so big that it’s doubtful we’ll run out of addresses even in a billion years. For now only a fraction of those addresses have been given out to public and even that’s a gigantic pool. Instead of octets IPv6 addressing uses 8 hextets separated by colons, for example, 2603:3005:6009:b400:0:0:0:61, also represented as 2603:3005:6009:b400::61.

The idea was that given enough time IPv6 will supplant IPv4 and while in 20 years there has been adoption, it has not been at the anticipated rate. Here’s a Google chart on the IPv6 adoption rate. IPv4 continues to dominate the Internet with technologies devised to extend its life, the main one being NAT.

Possibly the biggest hindrance to IPv6 adoption is that it does not interoperate with IPv4. Yes, they can co-exists in what’s known as dual stacking and they can communicate using translation technologies such as tunnels but they are separate technologies. Organizations have spent many years and resources to architect their IPv4 networks and they are understandably hesitant to spend the time and expense to repeat the process, especially considering the continued vast prevalence of IPv4.

Thankfully this site is not an organization with large investment in its IPv4 network so when I accidentally discovered that my Comcast Business modem was handing out IPv6 I was happy to start experimenting.

I have written before about the challenges of running this site on an old FC14 OS, but thankfully FC14 is IPv6 ready, no kernel patching or module loading necessary. With a bit of online learning I was able to spin up a public static IPv6 on the NIC and my server was now dual-stacked. As I progressed with the configuration I noticed that IPv6 has a bit of a learning curve even for the simplest operations. The ping command is ping6, traceroute is traceroute6, iptables rules no longer apply and one must use ip6tables, and there is no ARP table among other differences.

Spinning up an Apache website on IPv6 was done with a bit of trial and error but as I attempted to make the site public, I hit a road bump, a small indication of why IPv6 has been slow in growth. For years I had been using the free DNS service provided by my domain registrar, Network Solutions. But as I proceeded to add an IPv6 host name I noticed that Network Solutions provided no such mechanism. IPv6 records are added as AAAA rather than A records which are for IPv4 hosts. Reaching out to the Network Solutions tech support confirmed that they do not support AAAA records.

The only option was to move my DNS hosting to another provider that supported AAAA records, and that search led me to Hurricane Electric. I can’t compliment this company enough. Migrating DNS was easy and while their interface seems a bit dated and a bit cumbersome (there is no zone import facility) everything worked perfectly save a tiny glitch. Even more impressive, their tech support replied to my email within minutes with helpful answers to quickly overcome the glitch. I was impressed, and no, I am not getting paid to endorse Hurricane Electric, just a satisfied user of their free DNS hosting.

You can now browse to the IPv6 site and see for yourself, but you can only access it if your device is IPv6 capable. If your provider doesn’t provide IPv6 and you want to experiment with it, Hurricane Electric has a free service for that called Tunnel Broker for IPv4 to IPv6 translation. I tested that out on a Windows 10 host and it worked flawlessly.

Finally, if you want to see more details on your Internet connection, the whoami page will show you quite a bit of information about your online presence, IPv4 and IPv6 included.

June 8, 2018

The GDPR Mess

by robert hashemian @ 4:35 pm
Filed under: business,internet,law,web — Tags: ,

With GDPR (General Data Protection Regulation) being in full force since May 25, 2018, one must assume that the privacy and security of users are now fully protected. I think it’s an understatement to call that claim an over-exaggeration.

GDPR is a European regulation designed to protect the privacy of European citizens, giving them full control over their personal information. For most website operators it translates to getting users’ permission before doing anything with their data and deleting that data upon request.

While on the surface it is a well-intentioned law, little doubt remains that it has morphed into a giant confusion. Fact is no one really knows all the subtleties of this law and no one knows how to correctly implement it.

First there was a barrage of emails from companies proclaiming that they had new privacy laws, except that who has the time to click on every email and read reams of legalese nonsense.

Now we have the omnipresent ridiculous popup/slider on sites declaring some inane cookie policy for the site with a button to accept the terms. This site is guilty of that too, you might have noticed a cookie disclaimer sliding up from the bottom of the screen. The popup is just a utility script hosted on some site and I have no idea how it helps with your privacy and security while you are on this site.

Ironically, your privacy and security was just fine on this site prior to showing you the GDPR cookie notice. No data was being collected on you, no cookies were being stored on your browser, and no tracking was being done. Of course the Google services used on this site do some of those things and those are separately covered by Google’s privacy policies.

Now with the introduction of the cookie popup, this site has to use cookies to keep track of the fact that the user has been to the site and accepted the terms. In other words this site has to tell users that is uses cookie because it uses cookies to tell users that it uses cookies. And now the site hosting the popup code knows about the user too. Moreover the user that has just arrived to the site is not going to take the time to read all the cryptic nonsense in the privacy policy. Instead s/he is going to accept everything and continue. Now the site can do whatever it wants with the user’s data and it has explicit permission from the user. That provides a pretty strong incentive to abuse the data without any fear of legal consequences.

Finally, how does the European law expect a small time blogger provide the same level of privacy provisions of Amazon or Facebook to its users? Those are companies with billions of dollars at their disposal and an army of developers, attorneys and consultants.

Now comes GDPR with its esoteric rules to confound the small sites or even worse shut them down because they didn’t ask a silly question with a checkbox next to it. So much for democratizing the Internet where the small guys should have a shot at having their voice heard too.

But for now GDPR is here so by all means, read the disclaimer, visit the privacy page and click the stupid button. Don’t worry, your private data is safe with this site, especially since it doesn’t even ask for it.

October 22, 2017

The WHOIS Data Block

by robert hashemian @ 10:01 pm
Filed under: internet — Tags:

The WHOIS service is almost as old as the modern Internet. When you register a domain name, ICANN requires the domain registrar to collect the contact information of the domain holder and make that publicly available. There are a number of sites online that let users query the WHOIS database for various domain names. This site also has such a whois service.

Some registrars give their clients the option of private registration where the owner's contact info is replaced with generic data to protect the client's identity. That is fine except that some registrars like GoDaddy and Network Solutions charge fees for such a  service that cost them nothing to operate. Some ethical registrars such as Google Domains or 1&1 offer this service for free.

By ICANN rules The owner, administrative and technical contact information for the domain must be kept current which is why domain owners are contacted annually to verify this data and update it if necessary. There is also another ICANN rule that obligates registrars to make this information publicly available over a web page as well the whois service on port 43.

Here's where some domain registrars such as GoDaddy are in violation of the ICANN rule. A whois lookup for a domain registered on GoDaddy reveals that the domain's WHOIS information is delegated to whois.godaddy.com. Querying whois.godaddy.com for that domain over the whois port returns very little contact information for the domain when it should return complete contact information according to the ICANN rule. Instead it points users to GoDaddy's own website to get the domain details.

One must assume this is a GoDaddy scheme to get visitors to its site to peddle its products and services. But how does it get away flouting ICANN rule? So is it a rule or a suggestion? Some may approve of GoDaddy masking much of the domain contact info over WHOIS as a way to road-block spammers from speedy access. Of course this data can still be had on GoDaddy's own website. But if GoDaddy's intentions are to protect its clients from prying eyes, then why does it charge them for private registration that cost $0 to operate?

July 3, 2017

The Long, Hard and Possibly Foolish Path to SSL/TLS Security

by robert hashemian @ 10:57 pm
Filed under: internet,web — Tags: , , , , , ,

... or TLS 1.2 on Fedora Core 14/FC14 and other older Linux versions

With the chorus of secure browsing  getting louder and becoming more prevalent,  HTTPS migration is becoming inevitable. Going secure is a pretty major undertaking, fraught with numerous pitfalls. It starts with the source files that produce the html pages and it could get ugly if there's even one element in a page that is called over http rather than https, no green padlock in that case. The Protocol-relative URL (//) instead of the hard-coded http:// or https:// is quite helpful to that end. That's one of side of the equation. The other is the server itself.

I run my site on an older server with an old Fedora Core OS (FC14) and by extension an older version of web server software, Apache 2.2.17. Over the years I have updated a few components here and there and fixed and customized a bunch of others, especially after new vulnerabilities have popped up. Updating the server to the latest and greatest version would be a non-trivial task for me. The old hardware may not be sufficiently supportive, much of the OS customizations will be lost, migrating the data and config files will be a pain and there will be downtime as well. Yet FC14 cannot support the newer and safer SSL/TLS technologies considered acceptable by today's browsers.

At my day job, I have access to resources to overcome this problem by fronting the web server with other servers running newer technologies. For example a combination of HAProxy and Varnish provides excellent web acceleration, load balancing, and SSL termination without making any updates to the core web server. No such luck for a small time operator such as myself with limited resources, so what to do?

One approach would be to only upgrade parts of the OS and Apache (httpd program) that deal with encryption but there isn't much in terms of online resources dealing with this topic other than the customary advice to upgrade the OS. In the end this became a long process of trial and error but it was a successful endeavor with a good bit of leaning as a bonus. Here's how I did it.

Apache 2.2.17 running on Fedora Core 14 can be configured for SSL, however it can only provide support for up to TLS1.0, with older cipher suites and the weak RSA key exchange. I had already patched OpenSSL after the Heartbleed bug had become public but what I needed were newer version of libcrypto.so.1.0.0 and libssl.so.1.0.0 libraries used by mod_ssl.so, a module used by httpd to enable SSL.

I downloaded the source and built OpenSSL version 1.0.1u. Building applications from source code in Linux is usually a three-step process, configure, make, and make install. After 'make install' The new OpenSSL libraries were placed in an alternate directory, /usr/local/ssl, instead of overwriting their main system counterparts.

Next step was to incorporate the new libcrypto.so.1.0.0 and libssl.so.1.0.0 libraries into mod_ssl.so and my tool of choice for that was patchelf. I downloaded and built patchelf 0.9.

Before using patchelf I took one step which I am not sure if it were necessary in hindsight. That step was adding the location of these new libraries to a conf file under /etc/ld.so.conf.d and executing the ldconfig command to add this new location to the library cache /etc/ld.so.cache used by the linker.

Here's an example of a command I used to replace one of the libraries in mod_ssl.so, I did the same for the other:

./patchelf --replace-needed libcrypto.so.1.0.0 libcrypto.so.1.0.0 mod_ssl.so

Then I used the ldd command to make sure mod_ssl.so was now linking to these new libraries.

Following an httpd restart and checking one of the secure pages I had the encouraging green padlock on Chrome browser. Indeed the page was using TLS1.2 now with a strong encryption/cipher. Yet the key exchange was using the obsoleted RSA. The new OpenSSL libraries had certainly made an improvement to mod_ssl.so but the strong key exchange element was missing.

To overcome that issue I had to rebuild mod_ssl.so with a newer version of Apache source code. That was version 2.2.32. configure was done with the following parameters:

./configure --enable-mods-shared="ssl"  --with-ssl=/usr/local/ssl

And after make I found the new mod_ssl.so in one of the subdirectories of the source files. I skipped the make install step to avoid possible complications of the new version installing itself in the system. Interestingly the new mod_ssl.so was already linked to the new libcrypto.so.1.0.0 and libssl.so.1.0.0 libraries I had created in the previous step. I suppose adding that directory to the library cache had helped with that.

I placed mod_ssl.so in the folder to be loaded by /etc/httpd/conf.d/ssl.conf and restarted httpd. And it failed to start with a message about a missing symbol ap_map_http_request_error! Obviously mod_ssl.so couldn't call into this function of the older httpd (or some library) version.

To fix that error I edited the file modules/ssl/ssl_engine_io.c and replaced the line:

return ap_map_http_request_error(rv, HTTP_INTERNAL_SERVER_ERROR);

with

return HTTP_INTERNAL_SERVER_ERROR;

I admit, this is a blind alteration with possible adverse repercussions, so I don't vouch for it. Executing make once again yielded a new mod_ssl.so and this time httpd started just fine, this time with a strong key exchange added in. Testing the site with SSL Labs gave additional confirmation that SSL encryption was indeed working fine.

If wondering, I use Let's Encrypt for free SSL certificates. The recommended utility for obtaining certificates is certbot but that tool with its overly complex and finicky python virtual environment wouldn't work under FC14. The tool that worked beautifully was getssl. It's a simple and clean, yet powerful and flexible script written as one executable file in bash script. Kudos to the getssl team for creating this robust tool.

So there you have it for enabling modern SSL/TLS in an older environment, in this case FC14. The prevailing wisdom is to abandon the old OS and start off fresh with a newer platform. I don't disagree with that philosophy and I set out to do just that when I started on this journey. In the end, my way was possibly more difficult and more prone to pitfalls but ultimately it ended up being more satisfying and more instructive.

I haven't migrated the entire site to HTTPS yet, but you can click secure whoami to view and examine the first secure page.

 

November 6, 2016

HTTP to HTTPS Migration

by robert hashemian @ 10:26 pm
Filed under: google,internet,web — Tags:

https-ssl-tlsA universally secure internet may have its defenders and detractors but like it or not, Google is going to force site encryption (https) across the board.

First it was the SEO penalty threat, supposedly giving higher scores to secure sites but it doesn't seem like that worked out great. I think Google recognized that just giving prominence to secure sites over plain ones might lead to low quality sites stealing rankings from reputable ones simply by going encrypted. That would have meant poor search results pages, possibly alienating users and driving them to competitors such a Bing.

Now Google is coming at this from another angle, the Chrome browser and this one may stick. As Chrome has the biggest browser market share on the market, they can shame non-encrypted sites right from the browser rather than jeopardizing the Google search engine money machine.

Beginning January 2017 Chrome will print a timid 'Not secure' next to a plain page's URL indicating it is not encrypted. But that is just the start. The plan is to make the label bolder and more colorful with the future versions of Chrome. I suspect that at some future point Chrome may require users to jump through warning messages to show a plain page. That would be much like the cumbersome steps needed today to show a page when browsing to a secure page with a broken or invalid certificate.

The process of migration from a plain site to an encrypted site starts with obtaining a site certificate. This used to be an expensive proposition but nowadays a basic one can be had for free. In terms of the web server there are 3 ways to migrate a site from plain to secure:

1- In-place migration of the web server application - Just about any web server on the market today can handle secure connections as well as plain ones. The process generally involves installing the certificate, making some configuration changes and the site goes encrypted. Servers with multiple domains may however need an upgrade. For that, check for SNI support. For example Microsoft's IIS below version 8 does not support SNI. And if you have users that are still on Windows XP, good luck. SNI isn't supported on that platform at all.

2- Using an https appliance - Here the web server infrastructure is left intact but instead it is fronted by another server or service known as an https appliance or SSL termination. There are many such appliances on the market that are relatively easy to set up. There are also open source products such a Nginx or HAProxy that require a bit more tech know-how. In both cases they are deployed by installing the corresponding domain certificates and exposing them to the internet traffic. Internally they access the actual web server via plain http and return the page to the users encrypted over https.

3- Using a CDN - This is similar to the 2nd method, except that the appliance is actually managed by another company, like CloudFlare (free), Akamai or  CloudFront among others, in the cloud. The benefit is that little administration is required and in some  cases, like CloudFlare, even the certificate is pre-handled. The downside is giving up a certain level control and trust which a business may not be comfortable with.

Going https is not a trivial task, specially for the less tech savvy. But at least there are a number of available migration choices, each with a number of product options. They have various degrees of convenience, efficiency, and precision but eventually one must be chosen as the https migration seems inevitable. How would this site migrate to https? Remains to be seen.

... And now here's a glimpse: The Long, Hard and Possibly Foolish Path to SSL/TLS Security - TLS 1.2 on Fedora Core 14.

July 22, 2015

The Dawn of AWS Zombies

by robert hashemian @ 9:34 am
Filed under: hacking,internet — Tags: ,

awsOne of the less enviable tasks in a techie's life is identifying bogus robot traffic on their networks. Robots suck up bandwidth without giving anything in return and in most cases try to brute-force their way into systems and steal information and then assimilate their target hosts into new recruits in their army of zombie robots.

Identifying and neutralizing robots is hard enough, specially those hunting in packs causing DDoS headaches most of the time, but in past there used to be time, resources and funding barriers which moderated these attacks. With cloud services those barriers seem to have all but vanished and based on what I can see, AWS (Amazon Web Services) is one of the ugliest actors on the market. So how are AWS zombies created?

  • Hacker uses a stolen credit card to set up an account on AWS or hacks an existing AWS account.
  • Hacker spins up multiple virtual machines under this account. Or hacker breaks into a legitimate AWS virtual machine.
  • Hacker installs robot apps on one or more virtual machines and launches attacks.
  • Successful attacks bring more power to the hacker.
  • At some point AWS or the legitimate account holders notice high usages in processing, storage, and bandwidth and shut down the operation but by then the damage is done.

Could AWS be complicit in this type of activity? Perhaps not actively, but there is a passive element here as well. I'm sure they won't admit to it, but if a legit account is broken into and its cloud services are stolen, would AWS even care? They just blame the user for being careless and charge him for the usage. AWS may exercise more care in terms of blocking accounts with stolen credit cards because they may not be able to squeeze money out of those cases.

But even then, Amazon is so big with such vast resources that these cases may not even register as a blip on their radar. So the cycle of spawning AWS zombies will never cease and Amazon continues wasting our time, resources and bandwidth with impunity.

It may be an overkill to completely block AWS, but if partial blocking becomes necessary, a list of their public IP address ranges is published here and in json format here.

June 27, 2015

Craigslist eBay Motors Car Scam

by robert hashemian @ 1:09 pm
Filed under: internet,web — Tags:

ebay-craigslist-car-scamI'd been in the market for a used car when a too good price on Craigslist caught my attention. I'd sold a street bike on Craigslist a few years ago and had a good experience so figured to go into this but with raised antennas.

An email later, the seller reveals a sob story about the car belonging to her dead husband and wanting to move on. The car's in a great shape with all paperwork in order. Sounds plausible, so can I see the car? Seller replies the car is in some eBay garage across the country in lot number so and so.

No worries, she just needs my info and eBay will contact me about payment. The money will remain with eBay until I receive the car and I have 10 days to inspect it. If any issues, I can return it at no cost to me.

So I ask for the eBay page where the car is listed. Seller says she took it down because of the fees. But really, eBay will make all arrangements.

Yeah, sure man. Of course at this point the full blown scam was obvious, but should have been obvious at Craigslist. A quick Google search revealed that this unholy Craigslist-eBay alliance bait and switch is in fact very popular and a few people have been victimized, buyers and sellers.

So why this post? Just adding one more page to Google's search results to raise the warning volume slightly more.

Read this and stay vigilant. There's plenty more info on this. Just Google it.

December 22, 2014

DNS Verification Error

by robert hashemian @ 12:55 pm
Filed under: internet — Tags:

Recently it was brought to my attention that the domain name hashemian.com has a DNS error associated with it. The domain's DNS is hosted with its registrar as many registrars provide basic DNS service for free. This service generally consists of two name servers with varying degrees of restrictions to configure zones and records.

Concerned, I headed to dnscheck.pingdom.com to verify this for myself and indeed the tool does show an SOA records inconsistency error for the domain along with a couple of warnings.

dns soa error

The warnings are attributed to the fact that the name servers do not have reverse addresses (PTR records) associated with them. Having reverse addresses is not a requirement but it is recommended.

Having consistent SOA (Start Of Authority) records on all name servers is however required. Except that in this case the inconsistency reported was only due to upper and lower case differences between the records on the two name servers and that gets into a bit of an unknown territory.

According to this RFC document: Domain Name System (DNS) names are "case insensitive". That is stated right at the top of the document in the Abstract section.

Going by the RFC, Pingdom's DNS tool is incorrect in flagging the SOA records with differing letter cases as error. One can label the DNS management work at the registrar as sloppy or clumsy, but this discrepancy should at best only rise to a warning level.

December 7, 2014

Amazon Diapers To The Rescue

by robert hashemian @ 7:55 pm
Filed under: business,internet — Tags: ,

amazon diaperWhen I opened my inbox last friday, an email from Amazon greeted me with the title: "Announcing Price Reductions for AWS Data Transfer and Amazon CloudFront"

I wondered how Amazon was going to make up the difference in the face of stiff competition from other cloud vendors. Then I saw Amazon diapers in the news and all my concerns were laid to rest, genius.

Could this have been the starter line for the executive meeting at the Amazon's headquarters? "We have a revolutionary idea to counter the AWS shrinking margins and the Fire phone losses. Please bring your attention to the baby on the PowerPoint slide ..."

Revolutionary indeed 🙂

 

September 26, 2014

Linux Shellshock Bash Bug Workaround

by robert hashemian @ 12:55 pm
Filed under: computers,hacking,internet — Tags: , ,

The warnings about the shellshock bash bug are ominous and not unfounded. This is perhaps a greater risk than Heartbleed. Here are the gory details of this bug.

To test your system for this bug run the following command from the shell:

$ env x='() { :;}; echo vulnerable' bash -c "echo this is a test"

if you see the word 'vulnerable' anywhere in the output, like below, you have the bug.

shellshock bash bug

Because bash is such a fundamental part of Linux/Unix and used in so many ways and so prevalent, it wouldn't be that difficult for malicious hackers to use this bug to penetrate a machine and do all kinds of bad things including completely take over the machine. Web sites would the most obvious target of such attacks.

Now how to fix this. New bash versions with the bug patched have become available so users can update bash and be done. But this is not as easy to do for everyone. Some people may have older, obsolete versions of Linux, so they may not find the new patched bash version. They would need to get the source code and the patches, and then build and install it themselves. Yes, I know everyone should be on the latest version of everything, and I am guilty as charged, but let's dispense with the tarring and feathering for now.

Redhat however, in its haste and panic, had released a workaround on this page with a small block of C code that once installed, would disable function definitions and therefore mitigate this risk. They called it dangerous because one must assume this workaround would disable a legitimate feature of bash and possibly cause system failure if it were being used. Unfortunately a while later this workaround vanished (Update: actually here is the Redhat page for LD_PRELOAD mitigation. I don't know,  maybe the page never vanished at all. Just use the steps on that page then), but not before I had availed myself to it. For me, the ease and speed of its deployment made it worthy of a try. And here are the steps.

1- Put the following C code in a new file, bash_ld_preload.c.

#include <sys/types.h>
#include <stdlib.h>
#include <string.h>

static void __attribute__ ((constructor)) strip_env(void);
extern char **environ;

static void strip_env() {
  char *p,*c;
  int i = 0;

  for (p = environ[i]; p!=NULL;i++ ) {
    c = strstr(p,"=() {");
    if (c != NULL) {
      *(c+2) = '\0';
    }
    p = environ[i];
  }
}

2- Compile bash_ld_preload.c to get bash_ld_preload.so using the following command.

$ gcc bash_ld_preload.c -fPIC -shared -Wl,-soname,bash_ld_preload.so.1 -o bash_ld_preload.so

3- copy bash_ld_preload.so to the /lib/ directory like so:

$ cp bash_ld_preload.so /lib/

4- Add the following to the file /etc/ld.so.preload on a line by itself:

/lib/bash_ld_preload.so

5- Restart all relevant services or just reboot the system to be sure.

 

There you have it. I deployed this on several machines that run various applications. It killed the bug and there were no adverse effects. That means that those machines were not using the function definition feature of bash. Of course at some point we may write code or install applications that need to use this feature and if we have forgotten about this workaround, there will be a lot of head-scratching.

So, use the above workaround at your own risk. It will probably work for you, but the best approach as always is to update your platform and of course your version of bash.

Older Posts »

Powered by


Read Financial Markets  |   Home  |   Blog  |   Web Tools  |   News  |   Articles  |   FAQ  |   About  |   Privacy  |   Contact
Donate Bitcoin: 1K9TzBvQ2oaEb4tX9t2vKDtZouMcpfV6QF
paypal.me/rhashemian
© 2001-2019 Robert Hashemian   Powered by Hashemian.com