Monday 13 May 2013

[Build Backlinks Online] TITLE

Build Backlinks Online has posted a new item, 'Playing with a USB Missile Launcher'

This is the last half-finished “hairball” blog post about USB devices on Linux. I actually did manage to get a working program that controlled a USB foam missile launcher. Unfortunately, I didn’t document all the steps, so this blog post just sort of stops at some point.

I got a USB Missile Launcher for Christmas. The manufacturer, Dream Cheeky, provides software–but only for Windows XP. And I thought to myself, “wouldn’t it be fun to practice some USB reverse engineering skills?” Because another Christmas present was a USB protocol analyzer from Total Phase. I should note that plenty of other people have apparently already written drivers/software for USB missile launcher toys, but I wanted to poke around myself.

Total Phase makes a high-speed USB 2.0 protocol analyzer for $1200, or a regular-speed USB protocol analyzer for $400. Here’s a trick someone mentioned: if you get the cheaper protocol analyzer and need to work with a high-speed USB device, you may be able to plug the high-speed device into a low-speed USB hub to slow the device down.

I decided to start with ladyada’s excellent guide to hacking a Kinect by reverse engineering USB packets. So here’s what I did.

Step 1. Make sure the device works. It would suck to attempt to reverse engineer a broken device. I keep a Windows XP computer lying around, so I downloaded the software for it, installed the program, and plugged in the USB rocket launcher. After the install, XP wanted to restart, so I restarted the XP computer (unplugging my USB rocket launcher after the computer was off), then started the rocket launcher software back up, then plugged in the USB device. Sure enough, everything worked fine. The controls are: pan left/right, tilt up/down, and fire. Tip: the rocket launcher uses bursts of air, so don’t jam the foam rockets down hard on the launcher.

Step 2. Probe the device. I plugged the USB rocket launcher into a Linux machine running Ubuntu 10.04 (Lucid Lynx). I ran the command sudo lsusb -vv and the relevant info from the list of USB devices on my system was this:

  Bus 002 Device 045: ID 0a81:0701 Chesen Electronics Corp. USB Missile Launcher  Device Descriptor:    bLength                18    bDescriptorType         1    bcdUSB               1.10    bDeviceClass            0 (Defined at Interface level)    bDeviceSubClass         0     bDeviceProtocol         0     bMaxPacketSize0         8    idVendor           0x0a81 Chesen Electronics Corp.    idProduct          0x0701 USB Missile Launcher    bcdDevice            0.01    iManufacturer           1 Dream Link    iProduct                2 USB Missile Launcher v1.0    iSerial                 0     bNumConfigurations      1    Configuration Descriptor:      bLength                 9      bDescriptorType         2      wTotalLength           34      bNumInterfaces          1      bConfigurationValue     1      iConfiguration          0       bmAttributes         0xa0        (Bus Powered)        Remote Wakeup      MaxPower              100mA      Interface Descriptor:        bLength                 9        bDescriptorType         4        bInterfaceNumber        0        bAlternateSetting       0        bNumEndpoints           1        bInterfaceClass         3 Human Interface Device        bInterfaceSubClass      0 No Subclass        bInterfaceProtocol      0 None        iInterface              0           HID Device Descriptor:            bLength                 9            bDescriptorType        33            bcdHID               1.00            bCountryCode            0 Not supported            bNumDescriptors         1            bDescriptorType        34 Report            wDescriptorLength      52           Report Descriptors:              ** UNAVAILABLE **        Endpoint Descriptor:          bLength                 7          bDescriptorType         5          bEndpointAddress     0x81  EP 1 IN          bmAttributes            3            Transfer Type            Interrupt            Synch Type               None            Usage Type               Data          wMaxPacketSize     0x0001  1x 1 bytes          bInterval              20  Device Status:     0x0000    (Bus Powered)  

Note that my Vendor ID = 0x0a81 and my Product ID = 0×0701. Also note that bNumEndpoints = 1. An endpoint is a channel for USB data communication. Then we get the Endpoint info:

        Endpoint Descriptor:          bLength                 7          bDescriptorType         5          bEndpointAddress     0x81  EP 1 IN          bmAttributes            3            Transfer Type            Interrupt            Synch Type               None            Usage Type               Data          wMaxPacketSize     0x0001  1x 1 bytes          bInterval              20  

According to ladyada’s write-up, the “IN” means that data goes IN to the computer from the device, and the “Interrupt” transfer type is good for sending large amounts of small data quickly (e.g. a USB mouse).

Step 3. Prepare your Linux system to talk to the device. First, let’s review ladyada’s steps, which is for Windows. She installs libusb-win32 and then runs a program called inf-wizard to make a driver shell. Then plugging the device into Windows will attach the LibUSB-win32 device driver. Next, she installed Python and PyUSB.

I wanted to stick with Linux. I didn’t need libusb-win32 or inf-wizard.exe, and I already had Python installed. So my next step was to download PyUSB, extract the zip into a directory, change into that directory, then run sudo python setup.py install in that directory to install PyUSB. Since you’re installing PyUSB system-wide, you do need to run that command with “sudo” to run it as root.

Step 4. Write a short program on your Linux machine to talk to the device. I made a file missile-launcher.py and executable access with “chmod ugo+rx missile-launcher.py” next. Here’s the short program I ended up with:

  #!/usr/bin/python    import usb.core  import usb.util  import sys     # find our device  dev = usb.core.find(idVendor=0x0a81, idProduct=0x0701)     # was it found?  if dev is None:      raise ValueError('Device not found')    # Linux kernel sets up a device driver for USB device, which you have  # to detach. Otherwise trying to interact with the device gives a  # 'Resource Busy' error.  try:    dev.detach_kernel_driver(0)  except Exception, e:    pass # already unregistered     # set the active configuration. With no arguments, the first  # configuration will be the active one  dev.set_configuration()     print "all done"  

Note that my “idVendor=0x0a81, idProduct=0×0701″ parameters use the values I found from lsusb -vv. If you compare against ladyada’s short program you’ll notice one major difference. My code has these lines:

  # Linux kernel sets up a device driver for USB device, which you have  # to detach. Otherwise trying to interact with the device gives a  # 'Resource Busy' error.  try:    dev.detach_kernel_driver(0)  except Exception, e:    pass # already unregistered  

Ladyada’s PyUSB program for Windows didn’t have anything like that. But when I ran the program under Linux, I got the error message “usb.core.USBError: Resource busy”. It turns out that the Linux kernel tries to use a default kernel driver, and that prevents my program from talking to the device. Detaching the kernel driver lets me talk to the device just fine. I picked up this tip Ken Shirriff’s post about a USB panic button with Linux and Python. In theory you could also unbind the USB device from a command-line, but I prefer to do it right in my PyUSB program directly.

Note that you will need to run the python program as root, e.g. “sudo ./missile-launcher.py” or else you’ll get a warning message like “usb.core.USBError: Access denied (insufficient permissions)”.

At this point, you have a small working program that opens up a connection to the USB rocket launcher. If the USB rocket launcher isn’t plugged in, you’ll get a “Device not found” error, and if the USB device is plugged in, you’ll get an “all done” message and the program exits gracefully.

Step 5: Try to read from the USB device. In ladyada’s guide, she tried sending 0b11000000 = 0xC0 (“Read Vendor data from Device”) to a Kinect. I got no response from that, but I did get a response sending 0b10000000. That corresponds to sending:
- a ’1′ to read from the device
- a message type of ’00′ = standard. Ladyada got a response sending to ’10′ = vendor
- ’000′ (reserved bits, so always 0)
- ’00′ to say the recipient of the message is the device.
Then sending a request of 0 got back a result of two zero bytes:

  bRequest  0  array('B', [0, 0])  

Interestingly, running the same program again would get a “usb.core.USBError: Unknown error” response. At that point, I would unplug the USB device and then plug it back in to reset it. I didn’t get any other responses from trying to send message types of class or vendor (as opposed to standard), nor did I get any responses from try to send messages to the interface or endpoint (as opposed to the device). See ladyada’s guide for more details about fuzzing the device and what all the various bit fields mean.

Step 6: Set up the Linux computer to use the Total Phase Beagle. The CD worked nicely with HTML documentation on it. First, you copy some udev rules so that the device is writable by anyone when the Beagle is plugged in:


cd /media/Total\ Phase/drivers/linux/
sudo cp 99-totalphase.rules /etc/udev/rules.d/
sudo chmod 644 /etc/udev/rules.d/99-totalphase.rules

If you’ve already plugged in the Beagle, you’ll need to unplug it and plug it back in for these rules to fire. Next, you’ll need the Data Center software. You can get it off the CD, but I’d recommend downloading the latest software and user’s manual from the website instead. My CD had software version 4.20 for example and the website was up to 5.01. Extract the software zip file (either from online or the CD). Then follow the directions in the online manual (or user manual PDF). The directions according to the manual are

  1. Install the USB drivers and Data Center software. Copying the udev rules is enough for USB drivers on Linux. Unpacking the zip is all you need for the Data Center software, because the executable is self-contained.
  2. Plug the Beagle analyzer into the analysis machine. This was my Linux machine.
  3. Plug the Beagle analyzer into the bus to be analyzed. In this case, this was my XP computer. Don’t plug the USB missile launcher in yet though.
  4. Start the Data Center software. Run the program “Data Center” in the directory you extracted from the .zip file. Follow the rest of the instruction in the Quick Start section.

You may view the latest post at http://feeds.mattcutts.com/~r/mattcutts/uJBW/~3/LLA_S3A0Ncw/ You received this e-mail because you asked to be notified when new updates are posted. Best regards, Build Backlinks Online peter.clarke@designed-for-success.com

[Build Backlinks Online] TITLE

Build Backlinks Online has posted a new item, 'My wife keeps me grounded'

This is a harmless “hairball” post I had as a draft.

Me: Hey, they added me to popurls.com!
My wife: Never heard of it. (pause) Had you heard of it before?
Me: Yeah.
Wife: Really?
Me: Yeah!
Wife: (with an extra helping of sarcasm) Really?
Me: Yes!
Wife: (dripping with condescension) You’re a very important man.

You may view the latest post at http://feeds.mattcutts.com/~r/mattcutts/uJBW/~3/tIarGYZDVjs/ You received this e-mail because you asked to be notified when new updates are posted. Best regards, Build Backlinks Online peter.clarke@designed-for-success.com

[Build Backlinks Online] TITLE

Build Backlinks Online has posted a new item, 'Email backlog'

This is a “hairball” post you can ignore. However, this post does trace my thinking about how to scale webmaster communication. Part of me wants to start answering questions I get via email by stripping out the identifying information and then replying with a blog post. Instead of one person getting a single reply, everybody could see what the answer is.

I spent most of the past week tackling my horrendous email backlog. At the start of the weekend, I was just touching 500 unread emails. I got it down to 218 unread emails and 264 total emails in my inbox. Of course, the ones that are left are the harder messages. And out of those 264 emails, 167 are from outside Google.

A few weeks ago, I flew up to the Kirkland office for a couple days to catch up with the Webmaster Central team. At some point, we were talking about doing videos for webmasters. Someone said “Why don’t we just grab a video camera and see how many videos we can shoot in an hour?” So we did. We managed to tape three pretty informative videos in about an hour, and that includes set-up/breakdown time.

So now I’m looking at these 150+ emails from outside Google, and I’m pondering about how much time I should spend on email compared to other things. Email is a 1:1 communication, so I could answer 10 emails and help roughly 10 people. Or in the same amount of time, I could comment on a forum, start on a blog post, or plan out another video that could benefit a lot more people. I did a series of about 15 videos last year when my wife was out of town, and the videos have been watched over 300K times and downloaded over 100K times.

So to make a long story short, I’m trying to figure out how I should handle email going forward. I’ve been thinking about this for a while, but don’t be offended if I don’t reply to email as much going forward.

You may view the latest post at http://feeds.mattcutts.com/~r/mattcutts/uJBW/~3/pW8F3Ndxisg/ You received this e-mail because you asked to be notified when new updates are posted. Best regards, Build Backlinks Online peter.clarke@designed-for-success.com

[Build Backlinks Online] TITLE

Build Backlinks Online has posted a new item, 'What to expect in SEO in the coming months'

We just recently taped a new round of webmaster videos, and I thought this video deserved a full-fledged blog post. This is my rough estimate (as of early May 2013) of what search engine optimizers (SEOs) and webmasters should expect in the next few months:

Bear in mind that this is a very rough estimate, because priorities, projects, and timing can change based on a lot of different factors. But I hope this gives folks a ballpark idea of what to expect in the coming months as far as what my team is working on.

You may view the latest post at http://feeds.mattcutts.com/~r/mattcutts/uJBW/~3/kkkxIAY6MSA/ You received this e-mail because you asked to be notified when new updates are posted. Best regards, Build Backlinks Online peter.clarke@designed-for-success.com

Sunday 12 May 2013

[Build Backlinks Online] TITLE

Build Backlinks Online has posted a new item, 'One million steps'

Note: this is a “hairball” post and you can ignore it. I was so proud of my first one million steps with a pre-Fitbit pedometer. Now I’ve done 13 million steps and it’s just kind of normal. For the record, the Fitbit is a great little pedometer, but I tend to lose one every few months.

On a happy note, my pedometer registered my 1 millionth step not long ago. I started wearing it in June. This is what the downloaded step data looked like earlier:

One millionth step!

You may view the latest post at http://feeds.mattcutts.com/~r/mattcutts/uJBW/~3/K8sp7r0FDjU/ You received this e-mail because you asked to be notified when new updates are posted. Best regards, Build Backlinks Online peter.clarke@designed-for-success.com

[Build Backlinks Online] TITLE

Build Backlinks Online has posted a new item, 'Example debunk post'

Over the years I’ve written a lot of blog posts to debunk misconceptions or claims that weren’t true. Sometimes I publish the blogs posts but often I don’t. This is a pretty typical example post. Someone claimed that Google was evil for removing a particular domain, when in fact the domain had been removed from Google’s index via a self-service user request to our url removal tool.

When we see misconceptions, we try to figure out where the confusion happened and how to prevent that type of confusion in the future. It’s also safe to assume when you read “Google cancelled my account” stories that there’s usually another side to the story, even if for some reason Google doesn’t go into the details.

My guess is that you haven’t seen this one unless you live in Switzerland. A few months ago, a friend noticed this complaint in Heute Online:

Benbit complaint

My ability to read German is well, practically non-existent except for spammy words. So I asked a friend to translate it for me — thanks, Johanna. :) Here’s roughly what it says:

Search giant kicks Swiss blogger out of the index

"Google is evil after all"

Zurich – On his blog, Benbit* from Zurich often discloses security holes of big companies. This makes him unpopular (see box) – so unpopular that Google kicked him out of the index.

heute: Congratulations, you are one of the first Swiss citizens to be kicked out by Google. Proud?
Benbit: Nowadays, everybody uses Google. So, it is not funny at all if you suddenly disappear completely from the search engine. To me, Google's motto "Don't be evil" is not right. Google is evil and misuses its power.

Why did Google delete your site?
I don't have a clue. I sent emails and registered letters, but no one contacted me to give me reasons for this.

Might it be possible that this is connected to your hacker activities? Didn't you publish the security holes of many companies on your blog?
I did, but this doesn't violate Google's guidelines. I am neither a spammer nor have I been doing illegal search engine optimisation for my blog. My only explanation is that I stepped on the toes of a Google advertising client who in turn complained about me.

Any idea who this might be?
Well, one of the companies that I mentioned on my blog. Among them are also powerful major banks.

As a small blogger, do you have any chance at all against Google?
What Google is doing is a clear case of censorship and violates Switzerland's federal constitution. I demand from Google to provide me with information about the deletion from the index. Otherwise, I am also considering going to a justice of the peace.
* Name known to the editor. PS: Until our press deadline, Google did not comment.

http://blog.benbit.ch

Okay, let’s pause for a second. At this point in the story, I think we can all agree that Google is 100%, pure, concentrated eeeeeevil. How dare they squash that poor, hapless blogger at benbit.ch?

Except I haven’t told our side of the story. Our side of the story is pretty short: someone from benbit.ch used our automated url removal tool to remove benbit.ch themselves. Now why would someone from benbit.ch remove their own site (multiple times with multiple url patterns over multiple months, I might add), and then lay the blame at Google’s feet? I could speculate, but I genuinely have no idea.

One important thing to mention is that even with a really harsh story like this, we still look for ways to do better. For example, this incident happened in March of 2007 using our “old” url removal tool that had been up for years. In April 2007, the webmaster tools team rolled out a new version of the url removal tool. In my opinion, it kicks butt over the old tool in a couple ways:

1) site owners can easily see the urls that they’ve removed themselves.
2) site owners can easily revoke a url pattern that they’ve entered in the past.

Just to show you what I mean, here’s a snapshot where I’ve removed everything in the http://www.mattcutts.com/files/ directory of my site:

Url removal snapshot

As you can see, I can easily view the removal url patterns that I’ve submitted, and there’s a “Re-include” button if I decide to revoke a removal and start showing the urls again.

My takeaways from this post would be:

- Sometimes people say negative things about Google. Remember that there is often another side to the story.
- Even when people say negative things, folks at Google do listen and look for ways to improve. Case in point: the newer url removal tool resolves a whole class of potential misunderstandings like the one above.
- Google does provide a lot of useful tools for site owners. :)

I’m glad that the webmaster tools team works to make it easier to debug and to fix lots of issues for site owners. If the tool had launched just a month or two earlier, the folks at benbit.ch could have diagnosed their issue themselves — but at least everyone can benefit from the better tool now.

You may view the latest post at http://feeds.mattcutts.com/~r/mattcutts/uJBW/~3/zh7WWuP9wfA/ You received this e-mail because you asked to be notified when new updates are posted. Best regards, Build Backlinks Online peter.clarke@designed-for-success.com