A Fond Fairwell to my First TiVo

Back in 1999 I saw the light and bought a TiVo. Service for it was about $10 a month but thinking this would be a long-term appliance for me, I bought the lifetime service for $240. If I kept my TiVo for more than 2 years, I would be in the black.

That TiVo served me well. Over the years I hacked it by putting a network card in it and adding a second hard drive to expand its original 30 hour capacity to about 250 hours. I also added a web interface and a toolset that supported my TiVo to Podcast project that, as its name suggests, turned TiVo season passes into video podcasts.

Now, 10 years on, I am finally forced to retire my beloved TiVo that has served me so well. Today is the day that analog television transmission ceases in the United States. And as my beloved TiVo only has an analog tuner, the signal will go dark in just a few hours.

Now my back of the napkin math puts my monthly service at $2. Still very expensive for my cheap heart, but I think its respectable. Rest in peace, good 'ole TiVo. comments...

What About Peer to Peer Credit Cards?

It always seemed to me that traditional banks tend to make most of their money on the backs of people making small errors and having to pay penalties rather than just charging for capital as their business model might otherwise suggest. Credit card companies seem to be doing the same thing jacking rates to the maximum allowable limit if their customer goes late on a payment. All of this makes it look like the traditional banks and credit card companies are making money by being deliberately opaque, hiding their true cash cow of fees in the fine print.

The peer to peer lending market that has been springing up over the past few years (Prosper.com, LendingClub.com, etc.) has been exploiting the opportunity this opacity creates. Who would have thought these peer to peer lending sites had a chance against traditional banks? But borrowers and lenders warm up to them because they strive to become as transparent as possible which is a welcome change.

But why do I have to get a loan for a specific need? The reality of the situation is that the current peer to peer lending market is uncollateralized. I can't go hold a chunk of the house ransom if a borrower doesn't pay me back for that home improvement loan I gave him. I can't "repo" the motorcycle I helped that guy in Utah purchase if he defaults, so what sense does it make that these are specific loans for specific goals? I'm merely making uncollateralized loans to individuals for whatever purpose they want. (and they could be lying) Its really just a bet on the person's credit rating and "confirmed" financial situation.

So why couldn't we think of it more like revolving credit? Why couldn't I get a Visa or MasterCard from a P2P loan site instead? The P2P lending company would sponsor the process issuing a credit card to borrowers funded by lenders pooling their money. Adding to the existing P2P loan benefits, the lender makes more money because revolving credit interest rates are higher than "standard loans" and the borrower has much more clarity into the factors that can make his APR go down and credit limit rise over time.

Since there is no one purchase the borrower would be arguing for in their "loan application", their listing would be a generic revolving credit application. Lenders would fund slices of the pool the borrower uses as available credit and the APR and size of the credit pool would be the principal negotiating factors. Over time, as the borrower proved a good repayment reputation, a "credit review" would automatically go up for bid again potentially increasing his credit limit and decreasing his rate.

I think this would make the peer to peer lending market a little bit more realistic. P2P loans are generally uncollateralized just like revolving credit. Give borrowers an easier way to access their P2P credit by issuing credit cards and lenders will make more money without a change in risk. Uncollateralized P2P loans should be granted at higher rates than their traditional collatoralized counterparts. comments...

Movies Are Just Content

Sony Pictures CEO Michael Lynton was recently quoted as saying "I'm a guy who doesn't see anything good having come from the Internet, period." He explained the Internet has "created this notion that anyone can have whatever they want at any given time. It's as if the stores on Madison Avenue were open 24 hours a day. They feel entitled. They say, 'Give it to me now,' and if you don't give it to them for free, they'll steal it."

Now clearly there are positive aspects to the Internet and the head of a major motion picture studio is understandably somewhat jaded. I think, however, his explanation uncovers a more significant misunderstanding. Anyone having whatever they want at any given time is perfectly fine, that's why Home Depot is open 24 hours a day. But "if you don't give it to them for free, they'll steal it" is off the mark. People still go to the movies, pay for premium movie channels and rent DVDs all the time. If there is a reasonable delivery method with a reasonable price, people will use it. Maybe the store is too inconvenient or the price of a movie is too high.

Most content industries are bumping up against a new reality in the world. Rather than people going without your product if your price is too high, they just go around the market and get the content for free. So as a content producer, you are faced with the choice of doing what the customer wants, such as lowering your price and providing better delivery mediums, or go out of business. (That's not to say every download is a lost sale though.)

The music industry was brought to its knees over this years ago. Once a healthy industry, they started seeing people download MP3 files from the Internet. They started to go after the downloaders legally (as they should be defending their copyright) but missed the fact that the world had changed around them and the customer wanted some newer options. No longer would a CD store that closed at 9:00 PM be the way people bought music. Why can't the music store be open all the time? And why do we have to buy the whole album if we really only wanted one song? Rather than realize that the customer wanted options more in tune with their portable music player, the music industry started advertising campaigns pleading with customers not to download music. This went on until the music industry was almost completely gutted. With few options left and not seeing any significant decline in illegal downloading, they had to look outward for an answer.

It took a bunch of geeks at a computer company to walk in and literally school the music industry on the way the world now worked. Apple dragged the music industry, kicking and screaming, back to the money tree with the iTunes Store. If you don't really think about it, what possible sense does it make that Apple is in the music business?! They now rule the music industry because they understood what the customer wanted given what the new technology offered, and they gave it to them. Of course the music industry had their list of demands before coming to the money tree such as an insistence on copy protection. What customer actually wanted to limit the players their music played on? By now, of course, rationality has set in and copy protection has fallen by the wayside as well.

The movie industry is in a similar predicament. As technology advances, movie downloading becomes more and more realistic for the average consumer. The DVD market is going the same way the CD market went. I haven't met anyone that likes the fact that you can't skip over some of the advertisements on a DVD they purchased before getting access to the content. They just want to watch the movie! Can you believe people are still renting DVDs from Blockbuster and NetFlix? Like music, movies are just content that can be downloaded. That this isn't the norm for the average family household with a broadband connection is almost unfathomable!

Yet we still have the CEO of a major motion picture company missing the whole point of the Internet. Command and control using proprietary systems is out, the customer has options now. However tempting it might be to call the Internet just a den of thieves, the reality is that movies are just content and the rules for content companies have been rewritten. Ask any newspaper company. If you don't give consumers what they want, you will go out of business! The sooner Mr. Lynton realizes this, the more of the pie Sony Pictures will retain. comments...

Virtual Hadoop / Hbase Clusters on Mac OS X with Parallels and Fusion

Rather than pay the power bill on the 10 node Hadoop / Hbase cluster I was previously using for testing, I thought it might be better to recreate the cluster using virtual machines instead. I use a 64 bit Mac Pro desktop running Mac OS X 10.5.6, which can support 32 Gigs of RAM, so it seemed to make sense to give it a try with that.

In the world of Mac virtulization, there are two players battling it out for domination: the upstart Parallels Desktop and the incumbent VMWare Fusion. While Parallels was the first mover, which garnered it considerable users, Fusion entered the market with better features and more stability. Parallels came back with a better product for their next release, and the two have been fighting it out for dominance ever since with no clear winner.

I started with Parallels because it was the only option early on, and I had stayed with it continually upgrading through version 3.5. I hadn't upgraded to Parallels 4.0, the current version as of this writing, because most of the features were focused on Windows desktop users, and I was usually using Parallels for virtualizing Linux servers. So I started by creating a 4 node Hadoop / Hbase cluster in Parallels.

Rather than suffer through 4 Linux installs, I just made a master Linux instance and copied it out to create the 4 machines. Parallels assigns random MAC addresses to each instance so they are uniquely addressable over the network. I got a Hadoop cluster up and running, launched Hbase and started importing test data. Each node had one CPU, 2 Gigs of RAM (the maximum for Parallels 3.5) and 32 Gigs of storage. The import speed was fairly good, and if it were not for Activity Monitor, I probably wouldn't have noticed the 20% CPU usage when the cluster was idle. Everything seemed acceptable until I decided to arrange the Linux instances on another monitor. As soon as I did this, I started to get draw errors all over the place and, eventually, even after I had quit Parallels, I was forced to reboot my Mac. It was an avoidable situation but not an ideal one, so I took a look at VMWare Fusion.



(larger version)

VMWare Fusion had the benefit of the latest version and was also easy to set up. Like the last setup, I created a master Linux instance and copied it out, but, this time, I had 2 CPUs per instance. I didn't think this would add much, if anything, to the performance, but it more closely simulated the physical setup so I went with it. I ran the same versions of Hadoop and Hbase (0.19.1) as I had on the Parallels setup but got slightly better insert performance. The important exceptions were that Fusion consumed no noticeable CPU when the cluster was at idle, and the whole thing didn't crash my system when moving windows around. Parallels may have remedied these problems in their 4.0 release, but I haven't had the time to test that out.

Considering the 25 Amps it takes to run the physical 10 node cluster, simulating the cluster on one machine is a far better use of power if you can live without the raw speed. I can't tell you how many circuits I blew trying to balance the cluster with the other equipment in the server room! Of course, if you need the speed, you have to deal with the power penalty, but the setup within virtual machines allows me to develop code and simulate machine and rack failures without paying an outlandish power bill. I'm also able to test other distributed filesystems like Cloudstore and MapReduce frameworks such as Skynet. I've also tested up to 16 virtual nodes at the same time tough 10 is usually sufficient for what I'm doing.

If you have experience virtualizing Linux clusters, please leave a comment with your experiences. comments...

Broken eth0 when cloning Linux instances in Parallels / VMWare

When cloning virtual Linux instances, Parallels and VMWare pick a new random MAC address for the Ethernet adapter. The Linux udev system keeps track of MAC address to Ethernet device mappings in a persistent rule file located (in the case of Gentoo) here:

/etc/udev/rules.d/70-persistent-net.rules

If you clone a Linux instance, the kernel will boot and recognize the new MAC address, increment the eth0 device assigning the new MAC as eth1. That's probably not what you want to have happen. The easy fix is to just delete that file and reboot causing the udev system to rewrite the file with the new MAC as eth0.

Adding this to the list, when creating a master Linux instance that you intend to clone, you should blow this file away along with /etc/ssh/*key* so the newly cloned copies will find their eth0 and build a unique set of ssh keys.

Thanks to Vantoft for the hint on this. comments...

Trust Me: A Story about Re-Branding Orange Juice

I was looking for Tropicana orange juice at the store one day but they didn't have any so I ended up going with something else instead. The display case where it had been was filled with a store brand I hadn't seen before. I didn't like what I bought so the next time I went to the store, I went over to that display case again to get some Tropicana. Nope, still filled with some no frills store brand. So I decided to look a little closer. Usually the store brand isn't a very popular seller so not much of the display case is dedicated to it, but in this case the store brand was taking up about as much as a major brand like Tropicana used to! So I decided to look even closer. And this was the first time I saw it.

I'm a visual person. I really don't read if I think I know what it says. I've also built up a thick protective layer against advertising that almost nothing can penetrate. Its a self defense mechanism that I have built over the years to save me time. So when I'm standing in the supermarket looking at a display case full of advertising, I have to actively work to chop down that advertising veil that automatically goes up.

Low and behold, the store brand had the word "Tropicana" in a san-serif written up the right side with half of a picture of a very vanilla glass of orange juice to the left. I immediately decided I didn't want it because even if it was exactly the same orange juice, it wasn't the much more "contrast-y" old carton and would therefore only be half of the experience pulling it out of the fridge, pouring and drinking had been. Again I gave another brand a try which I liked more than the last.

Now that I was cognizant of the re-brand, I remember I went home wondering what the old brand had looked like. Whatever it was, it had much more color than what I had mistaken for a store brand. There was a picture of an orange I think. Maybe Tropicana was the one with the orange with the straw in it. I didn't remember.

The next trip to the store was far more rewarding. There it was, the old Tropicana on the shelf right next to the new impostor. I bought 3 cartons to stock up in case this was a fluke of some old stock they were just trying to get rid of.



As it turns out, I was taking part in a marketing fiasco that would result in a rare branding reversal for Tropicana. Those old Tropicana cartons I stocked up on were actually the new stock coming back in. The brand had suffered a roughly 20% drop in sales costing them around $33 million just because of schmoes like me. Merely 6 weeks after introducing the new packaging, Tropicana made the decision to revert back to the old design. read more...

Salary and Responsibility

Earlier in my career, I would never have believed how closely responsibility and salary level are tied. Not knowledge or even smarts, but responsibility. It doesn't matter how much you know or how smart you are, you ultimately get paid more for being more responsible. Of course other things do help but I've found the primary driver for reaching the higher end of your salary bracket is responsibility.

It is this way because generally you are either responsible or you aren't. Its not something you pick up. Those incapable of responsibility don't generally value it so they don't try to emulate it which creates self-imposed stratification layers. Given the age of shady business practices taking down large chunks of the financial system, the importance of responsibility becomes even more pronounced. Surprisingly, higher salary often comes hand in hand! comments...

The Financial Mess

Blaming the financial crisis on greed is a bit like blaming an airplane crash on gravity. Greed, like gravity, is a reality of the situation. comments...

Simplicity is Powerful

Why is Google so indispensable? Why is Twitter causing such an uproar? Why did Facebook virtually take over MySpace which had taken over where Geocities left off? Each is successively more restrictive than its predecessor.

There is also a negative cost to options and features. More isn't necessarily better. Sometimes it's just more. comments...

On Designers vs Engineers

I've given quite a bit of thought to the "gut feeling" of a design driven approach versus the "data driven" analysis of an engineering approach to design. But as happens every so often, I ran across someone (Scott Stevenson in this case) who has been able to clarify the issue in plain English. I quote:

"Visual design is often the polar opposite of engineering: trading hard edges for subjective decisions based on gut feelings and personal experiences. It's messy, unpredictable, and notoriously hard to measure. The apparently erratic behavior of artists drives engineers bananas. Their decisions seem arbitrary and risk everything with no guaranteed benefit.

Designers, though, are just as frustrated by the apparent blind allegiance to data at the cost of human experiences. They often feel as if engineers lose sight of the actual goal. Artists see data as a tool only, not a purpose onto itself. The reason for this is simple: data in isolation makes no guarantees about whether the correct thing is being measured, or whether the measuring itself is skewing the results."

That's exactly it. While I see designers and engineers frustrating each other all the time, the engineers always seemed to win because they have something quantifiable on their side. That statement there illustrates why the two generally don't see eye to eye while each has perfectly legitimate claims. comments...