This means that a node in your graph expected to get a variable, but was given a constant instead. This is a message you might see when trying to freeze a graph using graph_util.convert_variables_to_constants. You probably need to add 'y' to the variable_names_blacklist parameter when calling graph_util.convert_variables_to_constants so that it doesn't get converted into a constant. Or, if you're sure you want 'y' to be a constant then you'll need to figure out why your graph contains 'x' that expects it to be a variable; perhaps 'x' can be removed.
2017-04-19
Tensorflow tidbit: incompatible with expected float_ref
'x' was passed float from 'y:0' incompatible with expected float_ref
Tensorflow tidbit: What's up with :0 in the names of my variables?
tl;dr replace t.name with t.op.name to get rid of the :0
When getting the names of things in Tensorflow I've sometimes been confused by the difference between "x" and "x:0". It turns out that "x:0" is the name of a Tensor object, while "x" is the name of the op that generates it (such as Add or MatMul).
If you have a Tensor object t and you say t.name, you will get a name with ":0" at the end. If you want the name without ":0", don't just try to use string processing to remove the ":0"! Instead, you can access the op related to the Tensor with the "op" member variable, so you can get the name you want with t.op.name.
If instead you have an operation my_op and you want the name of its output tensor, you can say my_op.outputs[0].name.
When getting the names of things in Tensorflow I've sometimes been confused by the difference between "x" and "x:0". It turns out that "x:0" is the name of a Tensor object, while "x" is the name of the op that generates it (such as Add or MatMul).
If you have a Tensor object t and you say t.name, you will get a name with ":0" at the end. If you want the name without ":0", don't just try to use string processing to remove the ":0"! Instead, you can access the op related to the Tensor with the "op" member variable, so you can get the name you want with t.op.name.
If instead you have an operation my_op and you want the name of its output tensor, you can say my_op.outputs[0].name.
2014-06-10
Homebrew Oculus Rift Eye Tracker
 |
| No case, for now |
These trackers are trying to solve a harder problem than they need to, though. They're several feet away from the eyes, and they have to deal with arbitrary head motion and variable lighting. For virtual reality, we can fix all of these problems by building eye tracking into a head-mounted display. The camera can be fixed to the head and mere inches from the user's eye, in a completely controlled lighting environment. Accuracy can be much better!
Having eye tracking in a head-mounted display is attractive for several other reasons. It helps solve the VR input problem by adding a built-in input method roughly equivalent to a mouse pointer. There are a lot of possibilities for interesting game mechanics driven by eye tracking. It also helps solve the face-covering problem: when you're wearing a head-mounted display, your facial expressions are hidden. Eye tracking records the hidden part of your face, and combined with external cameras can recover your entire facial expression so that you can have natural conversations with other people in VR.
So, after the disappointing performance of the commercial eye trackers, I decided to try making a VR eye tracker of my own. Here's how I did it:
This diagram shows the setup. The gray area is the light coming from the Rift screen toward the eye. An infrared camera is mounted on top of the Rift and looks down toward a mirror placed inside the Rift between the screen and the lens. This is a "hot" mirror, which is transparent to the eye but reflects infrared light. With this setup, the camera gets a perfect image of the eye through the Rift lens, but the camera is invisible to the eye. It's as if the camera is looking through the screen directly into the eye.
Here again is what the camera looks like installed on the Rift. The mirror is mounted inside, invisible.
And here's what the camera sees! Check out some video here. Remember that the camera is totally invisible to the eye while it is recording this; you see only the Rift screen. This image is nearly perfect for pupil tracking (the bright reflections seen here can be problematic, but I've fixed that in a newer version by moving the illumination off to the side).
You can make this yourself too! Here are all the parts I used:
- PS3 Eye camera
- M12 lens mount for PS3 eye
- 8mm M12 mount lens
- Infrared filter plastic
- "hot mirror" (reflects infrared light, but transparent to visible light)
- Glass cutter
- Plastic snips
- Mounting putty
- Infrared LED
- Battery box
- Soldering equipment
- Disassemble PS3 eye.
- Replace lens mount.
- Cut a piece of the infrared filter to fit in the lens mount.
- Install 8mm lens.
- Disassemble Rift. You need to expose the screen, but you don't need to remove it.
- Cut a hole in the inner shell above the left lens.
- Cut a hole in the outer shell above the hole in the inner shell.
- Use the glass cutter to cut the hot mirror to fit in the Rift at a 45 degree angle.
- Use the mounting putty to mount the mirror inside the Rift and close up the inner shell (clean the screen first!)
- Use the mounting putty to mount the PS3 eye on top of the Rift looking down through the hole at the mirror.
- Solder the infrared LED to the leads from the battery box.
- Mount the battery box to the Rift.
- Insert the LED through the air holes and mount it on the outside of the left lens, pointing at the eye.
- Fine-tune the mounting positions of the LED and camera, and the lens focus.
After all that, how does it perform? I'm happy to say that accuracy is pretty good: With careful calibration I can get ±2 pixel accuracy on the Rift display. However, there's a big caveat: the calibration is extremely sensitive to the exact position of the Rift. Any abrupt head motion, or even changes in facial expression like squinting, will jostle the Rift enough to mess up the calibration and put the tracking tens to hundreds of pixels off.
Can the calibration issues be fixed? I'm not sure yet. The obvious solution would be to add some tracking of head motion so it can be subtracted from eye motion. However, there's a lack of stable markers to use for tracking. The eyelids obviously move around a lot, but even the surrounding skin can move quite a bit as you change your facial expression. The inside corner of the eye next to the nose may be the best tracking point, but it's difficult to see that far to the side through the Rift lens, and it will be a lot more challenging to track accurately than the pupil.
Another possible solution is retina tracking. In my current setup the LED is far from the camera axis, producing a relatively normal-looking "dark pupil" image. If a light source is placed very close to the camera axis, the retina reflects light directly back at the camera, producing a "bright pupil" image. This is the cause of the well-known "red eye" phenomenon in flash pictures. It may be possible to exploit this by focusing the camera on the retina instead of the iris, and tracking the motion of the retina directly. I'm not sure how well this would work, but how cool would it be to take pictures of your own retina?
Yet another possible solution is corneal reflection tracking. The LED produces not just one, but several reflections in the lens of the eye, which appear as bright spots in the image. For pupil tracking these reflections just get in the way, but with corneal reflection tracking they can be exploited. By tracking position of the reflections relative to each other and the pupil, the location and orientation of the eye can be found. This is how most commercial eye trackers work, and may ultimately be the best option.
I've had a lot of fun working on this project, and I intend to continue trying out new ideas. If you're interested in working on eye tracking for VR too, get in touch at jdarpinian@gmail.com!
2013-12-14
Pebble vs. Qualcomm Toq: Smartwatch showdown
There are a lot of new smartwatch contenders out there this holiday season. I've had my Pebble since February, and wearing it all year has given me a good idea of what's important in a smartwatch and what's not. The most heavily promoted smartwatch, Samsung's Galaxy Gear, has a fatal flaw: the screen isn't always on, making it mostly useless as a watch. The other big name smartwatch, Sony's Smartwatch 2, has been getting mediocre to poor reviews; but there's another contender that hasn't gotten as much attention: the Qualcomm Toq (pronounced "tock", apparently). It has a fancy new non-LCD screen technology and a lot of ambition, but how does it stack up to the Pebble?
I've had my Toq for a week now, and here's how I see it vs. the Pebble in the main important categories:
The Toq app's Bluetooth connection is more reliable than Pebble's too: it hasn't once disconnected from my watch. The Pebble app sometimes lies and shows that the watch is connected when it really isn't. Additionally, the Pebble app puts a persistent icon in your phone's notification tray, which is an annoyance if you like to keep that area clear. The Qualcomm Toq app doesn't show any extra notifications at all.
The Pebble's vibration motor is just loud enough to be annoying. The Toq is quieter, but still strong enough that you won't miss it.
Neither watch supports clearing your phone's notifications. Every time you dismiss a notification on the watch, you'll find popping up again when you unlock your phone. Dismissing notifications twice is kind of a bummer. The Toq is slightly better than Pebble here, with one-way notification sync: when you dismiss a notification on your phone, it will disappear from your Toq as well. That often doesn't happen on the Pebble, though notifications do time out after a while.
I never came close to running out of batteries with either watch, and I'd actually prefer a smaller watch with a smaller battery if it was available. I don't need my smartwatch to last many times times longer than my phone when I charge them both every night.
The Toq's screen is bigger than the Pebble's, and it has color, which seems like a big advantage on paper. In reality, though, the colors are difficult to see in most lighting conditions. Most of the time, the screen is essentially black and white.
The Pebble's screen isn't as exotic. It's an LCD, and it's strictly black and white (not even grayscale). But don't count it out yet! It's always-on and sunlight-readable just like the Toq. It's smaller, but that allows the whole watch to be smaller, which I count as an advantage. I also find that the Pebble's screen is sometimes easier to see in low light indoors; at certain angles the Toq's Mirasol screen reflects light in a way that makes it unreadable. The difference is slight, however. For now, Mirasol doesn't give the Toq any real advantage over the Pebble.
Both watches have a front light for viewing the screen in dark conditions. The Toq's light is brighter, but I actually find it too bright. When I'm checking the time in a dark movie theater, I don't want my watch to shine like a beacon. The Pebble will automatically light up when you get a notification, which is convenient. The Toq won't, so you'll have to manually activate the front light to see why your wrist just vibrated.
The Pebble doesn't have a touchscreen, but it does have physical buttons that click. The interface is simple and elegant. The tactile feedback of the buttons means that you don't have to stare at the watch while you're interacting with it. Also, the Pebble uses its accelerometer to implement a killer feature: one-handed activation of the front light. Simply twisting your wrist will light up the screen. This feature does sometimes trigger accidentally, but it rarely bothered me. In contrast, the Toq's backlight requires you to use both hands and double-tap on a hidden and unreliable touch sensor. Considering that you often need the front light to read the screen, this quickly becomes annoying.
The Toq comes with a few third-party apps built in, from "selected partners" like AccuWeather, E-Trade, and DoubleTwist. Frankly, they are boring. There is no Toq app store and no public Toq SDK. Until one is released you won't be able to customize your Toq in any meaningful way. The only bits of information you can display on the Toq's watch faces are the current time, the current weather, or one stock quote at at time.
Here's what I'd like to see from Pebble and Toq in the future:
Toq: Make it smaller (cut down the screen and battery if you need to). Keep working on that Mirasol display technology; it could turn into a real advantage with better colors and faster updates. Ditch the touch sensor buttons; put some real ones in. Release a public SDK. And for heaven's sake, use a standard replaceable watch band!
Pebble: Make your phone app more reliable and integrate better with the phone's notification system. Build a watch app store and feature some high-quality apps. Build some custom integrations with common apps: I'd love to see Google Maps Navigation instructions on my watch, or control my Chromecast with the Pebble's buttons.
I've had my Toq for a week now, and here's how I see it vs. the Pebble in the main important categories:
Phone app winner: Toq
Qualcomm has made a great smartwatch companion app. It is polished and provides an important feature that Pebble's app doesn't: the ability to send notifications from any Android app to the watch. The Pebble app only supports a few types of notifications by default, and although there are third-party apps that expand that support, their quality is questionable.The Toq app's Bluetooth connection is more reliable than Pebble's too: it hasn't once disconnected from my watch. The Pebble app sometimes lies and shows that the watch is connected when it really isn't. Additionally, the Pebble app puts a persistent icon in your phone's notification tray, which is an annoyance if you like to keep that area clear. The Qualcomm Toq app doesn't show any extra notifications at all.
Notifications winner: Toq
This is the killer smartwatch feature: being able to see who's calling or who just texted you at a glance, without even using your hands, is great. Both watches feature a vibration motor, and the best thing to do is to put your phone into silent no-vibrate mode, and rely on the watch vibration instead. The slightly better reliability of the Toq's Bluetooth connection is a boon here. The Toq will also notify you on the watch if the Bluetooth connection fails, which only happens if you leave your phone behind. The Pebble doesn't do that by default.The Pebble's vibration motor is just loud enough to be annoying. The Toq is quieter, but still strong enough that you won't miss it.
Neither watch supports clearing your phone's notifications. Every time you dismiss a notification on the watch, you'll find popping up again when you unlock your phone. Dismissing notifications twice is kind of a bummer. The Toq is slightly better than Pebble here, with one-way notification sync: when you dismiss a notification on your phone, it will disappear from your Toq as well. That often doesn't happen on the Pebble, though notifications do time out after a while.
Battery winner: Tie
Both Pebble and Toq have batteries that last 4+ days. Neither watch is small enough to consider wearing to bed. Since you'll be taking it off anyway, charging it every night is not a problem. The Toq is slightly more convenient with its cool wireless charger, but the Pebble's magnetic cable works quite well too. Neither watch requires fiddling with a USB connector for charging.I never came close to running out of batteries with either watch, and I'd actually prefer a smaller watch with a smaller battery if it was available. I don't need my smartwatch to last many times times longer than my phone when I charge them both every night.
Screen winner: Tie
The Toq's raison d'être is its screen. It's a testbed for Qualcomm's proprietary Mirasol technology, a potential replacement for LCDs. The screen is low-power, always-on, and readable in sunlight. In fact, the more light the better.The Toq's screen is bigger than the Pebble's, and it has color, which seems like a big advantage on paper. In reality, though, the colors are difficult to see in most lighting conditions. Most of the time, the screen is essentially black and white.
The Pebble's screen isn't as exotic. It's an LCD, and it's strictly black and white (not even grayscale). But don't count it out yet! It's always-on and sunlight-readable just like the Toq. It's smaller, but that allows the whole watch to be smaller, which I count as an advantage. I also find that the Pebble's screen is sometimes easier to see in low light indoors; at certain angles the Toq's Mirasol screen reflects light in a way that makes it unreadable. The difference is slight, however. For now, Mirasol doesn't give the Toq any real advantage over the Pebble.
Both watches have a front light for viewing the screen in dark conditions. The Toq's light is brighter, but I actually find it too bright. When I'm checking the time in a dark movie theater, I don't want my watch to shine like a beacon. The Pebble will automatically light up when you get a notification, which is convenient. The Toq won't, so you'll have to manually activate the front light to see why your wrist just vibrated.
Interaction winner: Pebble
The Toq has a touchscreen, which is cool. Swiping through the pages of the UI mostly works well, though the screen is slow to update compared to a phone. Unfortunately, most of your interaction with the watch will not be through the touchscreen, and that's where the Toq breaks down. The "menu" and "backlight" buttons are separate touch sensors hidden in the band itself. This was a terrible decision, because the sensors are unreliable, and you'll hit them by accident when you're fiddling with the band, crossing your arms, etc. Also, the touchscreen isn't active when a watch face is displayed; instead you're expected to swipe in the area below the screen, which isn't as sensitive.The Pebble doesn't have a touchscreen, but it does have physical buttons that click. The interface is simple and elegant. The tactile feedback of the buttons means that you don't have to stare at the watch while you're interacting with it. Also, the Pebble uses its accelerometer to implement a killer feature: one-handed activation of the front light. Simply twisting your wrist will light up the screen. This feature does sometimes trigger accidentally, but it rarely bothered me. In contrast, the Toq's backlight requires you to use both hands and double-tap on a hidden and unreliable touch sensor. Considering that you often need the front light to read the screen, this quickly becomes annoying.
Size winner: Pebble
The Pebble is much smaller than the Toq, though still on the large side for a watch. I much prefer the Pebble's form factor. The Toq's larger screen isn't a big advantage, and the bulky battery at the bottom of the strap is annoying.Watch Band winner: Pebble
The Pebble's band is a standard interchangable watch band, easily replaced with the band of your choice. The Toq's band is non-replaceable because it includes touch sensors plus a battery placed at the bottom of the band in a rather bulky and uncomfortable clasp. The band itself is not very flexible, and even worse, sizing the band to your wrist requires permanently cutting it. That's right: you must cut it to size with scissors. If you cut it too small, gain weight, or want to give the watch to someone with larger wrists, you're screwed.Durability winner: Pebble
The Toq is not waterproof. It doesn't even seem like it would be especially water resistant, though I haven't tested it. The waterproof nature of the Pebble is comforting when you're washing your hands or the dishes. The non-replaceable wristband of the Toq is also a disadvantage for durability. One small point in the Toq's favor, though, is that its flat touchscreen seems less scratchable than the Pebble's curved plastic face.Third-party support winner: Pebble
Pebble has put a lot of work into their SDK and there are many third-party apps available. They aren't always high quality, but you'll likely find a few you love, even in small niches. I personally like the Bitcoin watch face that keeps me up-to-date on the latest Bitcoin price. Installing apps is a bit weird, though, as you do it through third-party websites like mypebblefaces.com. The official Pebble app doesn't include any kind of watch app store.The Toq comes with a few third-party apps built in, from "selected partners" like AccuWeather, E-Trade, and DoubleTwist. Frankly, they are boring. There is no Toq app store and no public Toq SDK. Until one is released you won't be able to customize your Toq in any meaningful way. The only bits of information you can display on the Toq's watch faces are the current time, the current weather, or one stock quote at at time.
Price winner: Pebble
This is a no-brainer: Pebble is $150, Toq is $350. Being expensive isn't necessarily a deal-breaker for a watch; watches can be classified as jewelry and prices for luxury watches extend far into the stratosphere. However, neither the Toq nor the Pebble is attractive enough to justify their prices as jewelry.Overall winner: Pebble
Toq is promising, with some interesting ideas and a great phone app, but Pebble is a lot cheaper and does more.Here's what I'd like to see from Pebble and Toq in the future:
Toq: Make it smaller (cut down the screen and battery if you need to). Keep working on that Mirasol display technology; it could turn into a real advantage with better colors and faster updates. Ditch the touch sensor buttons; put some real ones in. Release a public SDK. And for heaven's sake, use a standard replaceable watch band!
Pebble: Make your phone app more reliable and integrate better with the phone's notification system. Build a watch app store and feature some high-quality apps. Build some custom integrations with common apps: I'd love to see Google Maps Navigation instructions on my watch, or control my Chromecast with the Pebble's buttons.
Subscribe to:
Posts (Atom)