Anil Gangolli
Collected Bits - Home Page and Personal Weblog
District Court curbs FBI secret probe power
Recent revisions of the Patriot Act have given the FBI power not only to force communication companies including phone companies and ISPs to turn over records without a warrant or any court authorization, but also to be able to force the companies not to tell anyone, customers or others, that they had revealed the information.
A district court judge has ruled that this violates the First Amendment and separation of powers, but is delaying enforcing the decision pending appeal.
From this New York Times article
Posted at 07:19PM Sep 06, 2007 in dogma | Permalink
Jingle Bells Oddity
This month’s Smithsonian Magazine (September 2007, p. 34) included a small blurb on the publication of the Jingle Bells song 150 years ago and this image of the sheet music cover
(used here without explicit permission):
There’s something awry about it. Do you notice?
Hint: Recall the lyrics. It struck me when the blurb indicated the original name of the song.
Posted at 11:01PM Sep 03, 2007 in System.out | Permalink | Comments[3]
Notes on gigabit ethernet, ubuntu, file transfers, samba
Over time, more of the devices on my home network have been becoming gigabit-capable. In practice this doesn't buy us much. Most of the time, we are all on the 54 mbps wireless net and most of the traffic is Internet browsing at much slower rates. So to date, I hadn't really paid much attention to what I'm actually getting from all of the gigabit hardware.
Recently however, with some video files and backup disk images getting stored on the home server, it has really become worthwhile to connect to the wired net for specific activities. The Mac laptops in the household have gigabit ethernet ports, and so does my development desktop box.
On a recent Samba transfer of a relatively short 200MB home video clip from my wired desktop to the server, I noticed that I was still getting rates consistent with only 100mbps ethernet, nothing near gigabit rates.
Here are some notes on my experiences trying to improve this.
The Setting
The client is running Ubuntu 7.04 (Feisty Fawn) on a homemade desktop box with an ASUS P5WDG2 WS PRO motherboard. The motherboard has two gigabit ethernet devices on it, one based on the Marvell Yukon 88E8001 and one based on the 88E8052 chipset. I'm only really using the first one (eth0 on my box). The drivers are the skge drivers. I'm not sure which version is in the kernel (2.6.20-16-generic) in the 7.04 distribution.
The server is running Fedora Core 6 on another homemade box with an ASUS P4P800-E Deluxe motherboard; it has one Marvell 88E8001 device onboard, and another gigabit device from a Linksys card which is the one on the internal home net. Both use the skge drivers as well.
These are both wired using 7-foot CAT 6 cables to a Netgear GS-105 gigabit switch.
First Problem Found: Autonegotiation on the Ubuntu desktop box
After poking around with ethtool, I find, hey, I'm getting 100mbps rates because eth0 on the Ubuntu box thinks it is running at 100mbps. Check the indicator lights on the switch. Yep. Hmm. Now what?
Unplug, replug, ifconfig eth0 down, ifconfig eth0 up. No luck. For some reason, the Ubuntu box is autonegotiating to 100mbps though both sides are 1000mbps.
My google searches provide no answer but suggest some others have similar issues; I've still found nothing that would suggest how to get the autonegotiation to work. The nearly identical situation with Fedora on the server is having no problems reliably autonegotiating up to the gigabit rate. Eventually, I resort to forcing the Ubuntu box to go gigabit by adding a pre-up line to my /etc/network/interfaces< file before the interface is brought up, as shown in the third line of the following fragment from this file:
auto eth0
iface eth0 inet dhcp
pre-up /usr/sbin/ethtool -s eth0 speed 1000 duplex full autoneg off
Activating Jumbo Frames
OK. Now that I get that working reliably, I do some perf tests with iperf running on both sides, using TCP_NODELAY, and otherwise taking iperf defaults, I get about 537mbps.
% iperf -N -c 192.168.1.2
------------------------------------------------------------
Client connecting to 192.168.1.2, TCP port 5001
TCP window size: 27.6 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.1.102 port 41548 connected with 192.168.1.2 port 5001
[ 3] 0.0-10.0 sec 640 MBytes 537 Mbits/sec
Much better! But I decide to check what I get by activating jumbo frames which my hardware all supports.
With jumbo frame support enabled I get:
% iperf -N -c 192.168.1.2
------------------------------------------------------------
Client connecting to 192.168.1.2, TCP port 5001
TCP window size: 73.6 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.1.102 port 51573 connected with 192.168.1.2 port 5001
[ 3] 0.0-10.1 sec 923 MBytes 764 Mbits/sec
That's more like it. I think this is about the best I am likely to do, and in any case I decide I like 764 much better than 537
, so I add another pre-up line to set the MTU on eachreboot. My /etc/network/interfaces entry for the interface now looks like this:
auto eth0
iface eth0 inet dhcp
pre-up /usr/sbin/ethtool -s eth0 speed 1000 duplex full autoneg off
pre-up /sbin/ifconfig eth0 mtu 9000
Samba, Nautilus still slow
OK. Now that I've got my network speeds up I should be getting great transfer speeds, right?
Not quite.
Back in Nautilus (the GNOME finder GUI) I try dragging the 200MB video clip across, and I find
it still takes about 20 seconds, or 10MB per sec or roughly 100mbps again.
Could disk speed be the bottleneck? I don't think so; I've got fast SATA drives and they shouldn't
be a problem. To verify I try a little ftp test:
ftp> get Garden.dv
local: Garden.dv remote: Garden.dv
200 PORT command successful. Consider using PASV.
150 Opening BINARY mode data connection for Garden.dv (219360000 bytes).
226 File send OK.
219360000 bytes received in 2.23 secs (96182.1 kB/s)
OK! Now that's what I expect, roughly the speed of I'm seeing with iperf. So my desktop transfer problem must be a Samba problem right?
I go check my Samba server config, and consulting various sources on the
net, I find I already have the options set pretty well:
socket options = IPTOS_LOWDELAY TCP_NODELAY SO_RCVBUF=16384 SO_SNDBUF=16384
So, what about straight Samba transfers? Let's try smbget:
% /usr/bin/time -p smbget -w BUSYBUDDHA -a smb://sycamore-int/public/Garden.dv
Using workgroup BUSYBUDDHA, guest user
smb://sycamore-int/public/Garden.dv
Command exited with non-zero status 1
real 4.35
user 0.20
sys 1.01
The nonzero exit status turns out to be spurious. The file has transferred fine; took twice the time of FTP, but still not bad. Close to 500mbps. A similar test with smbclient yields similar results for me. Both of these are far better than what I'm seeing with Nautilus.
After more searching on the net, I decide to try smbfs for comparison
# mount -t smbfs -o username=notshown,password=notshown,sockopt=TCP_NODELAY //sycamore-int/public /media/public
This mounts it on /media/public and then using straight cp I get a 12 second transfer; slower than smbget.
But then I try a second transfer, and I get:
/usr/bin/time -p cp /media/public/Garden.dv .
real 0.95
user 0.02
sys 0.90
Great! Less than a second. Wait. That shouldn't really be possible. Check the md5sum. The file is all there. What is going on? Oh. Some sort of caching. Force the cache to be blown by remounting and you get the 12 second transfer again. So smbfs might perform really well for situations where you get to use the cache, but based on my limited experimentation, not for initial transfers. [Note: I also tried this with mount -t cifs and got the
same kind of behavior.]
Faster than Nautilus which is amazingly achieving only about 100mpbs still. So I'm still looking for improvement.
Summary
In summary, for my particular situation I found:
- Check that your ethernet device negotiated to 1000mbps. I had to force mine.
- If your hardware supports it, use jumbo frames. This got me to a measured 760 mbps using iperf (TCP) which I feel is probably as good as I'm going to get.
- FTP is able to achieve nearly the transfer rates that I see with iperf, and Disk I/O is included.
- I was able to get Samba doing about 500mbps via smbget or smbclient. I suspect this is about as good as I'll get.
- Using smbfs mounts is slower on initial transfers than smbget or smbclient, but fast on cached stuff.
- Samba transfers via Nautilus are still only getting transfer rates close to about 100mbps. I still don't know why Nautilus is so darn slow.
Tagged: ethernet linux gigabit samba ftp ubuntu
Posted at 01:59PM Jul 22, 2007 in tech | Permalink
Restoring habeas corpus
At last, the Fourth District Court of Appeals has ruled that the government can’t continue to hold people under military detention indefinitely without right of habeas corpus simply by calling them “enemy combatants”.
It was a 2-1 decision and it’s likely to go up to the Supreme Court. Let’s hope it holds.
Posted at 02:37PM Jun 11, 2007 in dogma | Permalink
Scary Report Card
It’s nice, but scary, to see a bit of honesty about the current situation in information security in government agencies .
The problems are deep and pervasive, and progress is slow.
While one can understand but not excuse an F overall for the entire Defense department, it sure is hard, as you file all of that information to the IRS, to notice that the Treasury Department gets an F, slipping from a D- last year.
Talk about a honeypot.
Tagged: technology security
Posted at 08:17AM Apr 13, 2007 in tech | Permalink
Hats off to the clip
Most people on the street will not recognize the specific utility of the item above.
You, of course do.
I think it is time we acknowledged the importance of this time-honored device.
Some historical events related to the clip and attempts to replace it:
- ca. 1985
- Apple ships a small plastic pin with Macintosh, hoping to alleviate users from constructing crude tools like the clip. Most users throw it away not really understanding what it is meant for, only to be sorry somewhat later. The net result is actually a significant increase in clip-based implement production and usage.
- ca. 1996
- Palm ships a plastic stylus with its Palm Pilot device, the tip of which can be screwed off to reveal a clip substitute. User adoption rate of this feature is low, with most users still hunting for their clip. Clip usage continues to rise as Palms and competing devices proliferate.
- ca. 1997
- The Office Assistant, a derivative of Microsoft's Bob project, makes its appearance. Its default avatar is an animated paper clip. While clearly an attempt to parlay familiarity with the implement that Windows users use daily with their machines, it is flawed in one important respect; no extended prong. The clip avatar and the Office Assistant feature in general meet with public disdain.
Despite these affronts the clip seems to be as popular as ever. So we tip our hats to the trusty clip.
Posted at 04:11PM Mar 25, 2007 in System.out | Permalink
Workaround for Firefox on Vista screen jitter
I was seeing a lot of screen jitter on Firefox (I’m on 2.0.0.2) running on MS Vista.
The workaround, quoted from mozillazine forums here is to disable the Bookmarks Toolbar from the view.
View | Toolbars |(uncheck) Bookmarks Toolbar
Seems to work.
Tagged: firefox
Posted at 08:48PM Mar 22, 2007 in System.out | Permalink
Upgraded to Roller 3.1
I’ve upgraded the site to use Roller 3.1. Note, I’m runnning RC6 --the official release is not yet out, and at least one more release candidate is expected.
Because of all of the model and macro changes, I’ve adopted a simple theme which I hope to improve over time, probably in a different direction than my old theme anyway.
It’s a bit ragged, but the content is there.
Possibly due to the change in permalink structure, my blog entries are all thought to be new by java.blogs which somehow got me on the hot blog list inadvertently.
Not much to see here. Certainly not much new. Sorry for leading readers astray.
Tagged: roller
Posted at 05:59PM Mar 18, 2007 in System.out | Permalink
Feeling small in the "Universe for Dummies"
Physics Nobelist George Smoot, colleague Saul Perlmutter, and the preponderance of “dark matter” are featured in this NY Times Magazine article by Richard Panek.
Posted at 08:16AM Mar 12, 2007 in System.out | Permalink
Upgraded to Fedora Core 6
As a result of the recent timezone data upgrade, I had noticed that my server box which was still running Fedora Core 3 had really gotten way behind the yum support curve. So after the DST switch went fine, I was feeling that the stars were aligned for mucking with my server box and I decided to upgrade the OS to Fedora Core 6.
These decisions are always about as likely as not to lead to an extended trip through IT hell, but it all went surprisingly well. I had to fuss around with a few configs that had changed formats, but nothing too bad.
As a result of the upgrade, I’m also now running MySQL 5.0.27 right from the FC6 RPMs.
Posted at 04:00PM Mar 11, 2007 in System.out | Permalink
Check your linux timezone updates
I was feeling comfortable that I had up-to-date tzdata installed on my linux box
rpm -q -i tzdata
showed I was running
tzdata-2005r-3.fc3
So, good; I've got the 2007 rules.
Not so fast!
zdump -v /etc/localtime | grep 2007
gave a transition time in April still. What's up?
It turned out that the rpm update had left /etc/localtime untouched and created /etc/localtime.rpmnew. When I moved that into place, it gave the right March transition, but it was timezone info for the Eastern US time zone.
I ended up having to copy my local America/Los_Angeles zone data manually from /usr/share/zoneinfo/America/Los_Angeles to /etc/localtime and then reboot.
This seems to have done the trick. I'm not sure how my tzdata installation went astray originally, but you may want to check boxes you have.
Posted at 12:27AM Feb 22, 2007 in System.out | Permalink
How doctors think
House, M.D. fans (like me) may enjoy this article on how doctors think by Jerome Groopman in the New Yorker magazine.
Posted at 09:19PM Feb 14, 2007 in System.out | Permalink
Self-referential formula
A friend at work pointed me at this self-referential formula , which I thought amazing and worth spreading, though I’m probably the last person to hear about this.
Posted at 09:20AM Feb 14, 2007 in System.out | Permalink | Comments[2]
Taking the high (profit) road
It is not often that taking the high road and taking the high profit road coincide, but Steve Jobs has found one of them in his recent letter (Thoughts on Music, February 6, 2007 ) in which he advocates for DRM-free content.
This is a win-win all around.
It’s pretty clear why it is in the consumer’s interest, well, at least to the savvy consumers anyway.
The Jobs letter makes no attempt to hide why such a position is in Apple’s interest, and how maintaining the FairPlay DRM system they have now is so onerous and not readily opened up for wider use (if any other digital download-based retailers would actually want to adopt it anyway). Not evident from the letter is Apple’s motivation from the rise of competing DRM-free distribution services like eMusic, but that’s ok, because these competitors would also benefit from greater selection in a DRM-free environment.
So now it is just a (simple) matter of convincing the music companies.
What’s in it for them?
* Lowering distribution costs and allowing more uniform distribution mechanisms without individual unit license management.
* Not alienating the large honest part of the population that’s going to buy (not steal) their music in the first place
* Huge potential volume growth (at the aforementioned fraction of their current distribution costs).
Will they get it? Probably not. Not until consumers like us and digital distribution channels like Apple and eMusic unify to pound it into their heads. Apple’s position statement is a step in the right direction.
Posted at 11:45AM Feb 11, 2007 in dogma | Permalink | Comments[1]