Back when Apple was looking over Blekko this was something we tried to point out to them (and the Siri team). You can't dialog what you don't know. Indexing the web really helps with that because not only can you pick up a lot of documents you can get a sense of which ones humans think are more important than others. To this day I don't know why Apple doesn't have a search engine team and crawler.
Interesting, but no info on the set of 3000 questions. For example, maybe Alexa is better at the top 100 most asked questions, but Google Home is better at the long tail.
Odd that there wouldn't be more insight either there, or at the 360i site.
As an owner of four Echo / Echo Dots in my house, I can say that Alexa is terrible at answering questions outside of a few common areas (weather, store times, etc).
With that said, I think the true power of Alexa's ecosystem is it's smart device integration. I'm sure Google will catch up soon enough, but at the moment I feel like Alexa still has a sizable lead.
Maybe I'm an outlier, but turned out I don't care at all for asking home assistant various questions. I have GH and my usage is mostly limited to "turn on/off lights" and "set timer". It's just that moments like "we need to ask what movie someone also starred at" are not really a part of any conversations we have at home, and if we really need to look some info up we will do it looking at screens.
Google Home being ridiculously bad at how it works doesn't help either - it feels like it's getting worse every week. I hate it how it understands me from fifth attempt now, I'm tired of hearing "I cannot help with that" 3/4 of the time for the same command or just listening for a minute and then do nothing. And it's not a matter of dusted mics. And not even starting on all that nonsense as declaring company name many times a day as a wake word, still absent reminders and frigging fiasco with shopping lists. Yet another "let's start with first iteration of product and then move to something else leaving it half-baked, because ADHD and we can"
The problem with asking these things questions when you are in a group of people is that now the whole conversation pauses to hear the damn answer. And then there is the 5-10 second processing time and the very slow non-interactive answer. It's just so much more efficient to have one of the people in the conversation whip out their phone and use Google, then say the answer when it's ready. Especially if the answer is not "Who was the start of The Martian" but something like "How closely was Lincoln the movie based on historical events?"
I will say, I also use my main Echo to play music. It is really nice that way, though song, artist, and playlist name recognition is atrocious. "Play Top Hits" and "Play Today's Top Hits" produces completely different result. Don't you know what I mean Amazon? Don't you know what I mean?
My kids (turning 4 soon) have figured out how to use it too, which is kinda neat.
> and then there is the 5 - 10 second processing time
Almost everything you've described matches my experience with the Echo exactly, but if there is one thing that it is consistently good at for me anyway, it's seriously low latency in responding. I've never observed anything like 5 seconds to process my input, let alone 10. It's been way better than Siri in this regard.
Own all of them, any time one fails, try the others for comparison. Over time, I find Alexa “most improved” and Google most frequently laughably terrible. (Siri most integrated, but most useless outside her zone.)
IOW, Google Home might answer most questions but is also most likely to just cross its fingers and pray the upcoming nonsense is useful.
Try asking straightforward questions like “Who was the oldest President?” or “Who won last year’s basketball finals?” to see these patterns.
>at the moment I feel like Alexa still has a sizable lead.
Do you really? I have 4 Echo & Echo Dots too and I love them, but the level of smart home integration is incredibly basic... "turn on the X (light)", "set the X (light) to 50%", "set the heat to 20" and "trigger find my phone".
(edit: this post sounds pretty negative, it's mostly frustration because the speech recognition is actually really good, so when it fails it's really noticeable & the shallowness of comprehension of what I'm actually saying to it is so shocking when the speech recognition tech is that good)
It will fairly often mishear light names (like my bedroom light, it confuses my name with "Theatre"), the integration will fail to contact a light (and instead of retrying the echo just says "I couldn't contact light X". It doesn't understand the room it's in (so I'm constantly having to name lights, rather than just tell it to turn on the light, or dim it).
I feel like the level of integration needed from people like Hue and Nest is so simple that even the most basic integration into a competitor (with just as good / better mic+voice tech) could blow Alexa away overnight... it actually feels like something you could build in a few hours with access to the Alexa processing repo.
...which really kills me, because as you say Alexa is better than anything out there, but it feels like all they're doing is adding funny responses to pre-canned phrases rather than trying to seriously improve their smart home proposition or trying to tackle the context/discoverability problem
> It will fairly often mishear light names (like my bedroom light, it confuses my name with "Theatre"), the integration will fail to contact a light (and instead of retrying the echo just says "I couldn't contact light X". It doesn't understand the room it's in (so I'm constantly having to name lights, rather than just tell it to turn on the light, or dim it).
We tell our customers to put their alexa devices into groups in the alexa app. It tends to help your situation quite a bit. For instance for our Lyric thermostats the default name assigned is "Lyric". So you can imagine the things that happen when telling your Alexa to "Make Lyric Warmer". Haha.
That being said, the challenge for them is "Making a light a light, and making a thermostat a thermostat" and knowing every time when someone says "turn on X" they're talking about that thing.
I've noticed my Google Home does a better job at being 'fuzzy' in that regard.
>We tell our customers to put their alexa devices into groups in the alexa app. It tends to help your situation quite a bit. For instance for our Lyric thermostats the default name assigned is "Lyric". So you can imagine the things that happen when telling your Alexa to "Make Lyric Warmer". Haha.
I'm not sure I understand - are you saying there's a way to put an Echo into a group so it understands where it is in some way? I don't see that option in the app anywhere, but maybe I'm just missing it!
If you mean to put the devices into groups so you can control multiple devices at once or come up with names that the speech recognition doesn't misunderstand, I do that - but at a certain point I gave up because the names would have no connection to the actual lights (my bedroom light is my name, which is a pretty common name - but the Echo mishears it pretty regularly). I can't help but think that for my case when Alexa knows I'm talking about a device & no device matches the understood text it should look at the other possible interpretations of the voice data to see if any of the likely ones.
The majority of my problems come from the lack of basic contextual understanding of where the Echo device exists in the house, which is a much more solvable problem than trying to consider multiple alternative interpretations of a device name!
I'd be willing to bet, the speech recognition is tuned to recognize common vocabulary over proper names. So it is purposely choosing "theatre" instead of "Peter?".
I've name my bedroom light "bedroom". So.. turn on "bedroom". Or turn on "table", turn on "window".
It will probably recognize something like that more consistently than your name.
>I'd be willing to bet, the speech recognition is tuned to recognize common vocabulary over proper names. So it is purposely choosing "theatre" instead of "Peter?".
Yeah, I think that's the case: it will usually get "Peter" but maybe 3 times a week it hears "Theatre". The problem is we have multiple bedrooms - so we'd need to come up with unique but common and unambiguous names that also feel natural to each person (which is the hard problem - we have 13 hue lights, 6 smart switches, a nest & a few software defined switches that do things like send commands to the TV to switch to specific apps)... or just add a new device group for each common mispronunciation and call it a day.
The reason I'm so insistent about Alexa understanding more about context is that within a room there are obvious, common vocabulary short names for devices, but coming up with a natural global naming structure within a house is less obvious (we experimented with calling the bedrooms front/middle/back but that phrasing isn't natural)
Basic room contextual information would also make for a really natural configuration UI within the app: you create Rooms, you add your Smart Home Devices (optionally assigning each a name within the room) and your Echo to a room. When matching against a "(Turn|Switch) (on|off) :device" expression, it can first match against room-specific names and then fall back on global names.
Yeah, OK I understand your meaning a little more now.
That's one of the nice differences between Google Home and Alexa. You give your Google Home a location, and when you connect devices it asks you to assign the devices to a room.
So you can say "turn on bedroom lights" and any light assigned to the bedroom it will turn on.
To follow onto my original comment, placement of the device also seems to matter, and how fast you talk or whatever accent you have seems to impact it as well. At least from an Alexa standpoint.
Speaking from experience, the integration model for Smart Home into Google Home was a really time consuming experience up until recently. Now it compares to Alexa from a level of effort perspective.
In reference to the OP topic, I'm curious... How often do you use your echo as a Q&A device? Or is it just for media and assistant like functions (alarms, shopping, etc)?
Why the downvotes on this? I appreciate the thoughts. I think it's a real shame that Google's assistant isn't available in a desktop client, that can use either text or voice input.
It might be just a visceral, reflexive reaction to reading "neuralink brain wi-fi device" and then thinking to oneself, "am I really ready to join the Borg collective?"
I've been working with speech systems on and off since 1998. They're all pretty bad in my opinion. They're not even almost there yet.
I remember studies from years back that speech is far lower throughput than typing for nearly every language except Chinese.
I hate talking, especially to a computer, when I have a typing interface. Speech is such a serial, synchronous information channel. That fact paired with how context-limited speech systems are makes for a very poor experience. It's really going to take an inventive design to get people to use speech regularly for machine interaction. We're just not there yet.
Well that's not surprising. Google basically ported over Google Now, a mature product; and Google has data. Amazon had some shopping data, and that was it.
Without seeing the 3000 questions and what categories they fall in, I wouldn't put too much stock into this article. I use both on a daily basis and Alexa is far more accurate than Google Home, but again that's my use case, mostly asking for radio stations, songs, news trivia questions and simple arithmetic.
I asked Google Home: "Is it cold outside?" and it responded: "Sorry, I can't answer that yet." Seriously!?
This is the same for all of these home devices, but I have to talk to it in such a robotic manner, and if I deviate at all it won't do what I want. It definitely has some niceties, but a long way to go to being essential in my life.
That's weird; the Assistant on the Pixel responds to that with "No, it's not cold in Mountain View right now. The temperature right now is 29." (and shows a weather forecast card).
I can confirm both of these results: Google Home said "Sorry, I don't know how to help with that" and Google Assistant on my iPhone answered "No, it's not cold in Seattle right now. The temperature right now is 66."
Yes, Alexa is bad at answering random things we throw at it, but making an app to control what I want to and how is why I love it. I don't trust Google to be as open.
This headline reminds me of 80s and 90s when folks touted why <pickYourOS> was better than windows. Ultimately, the ecosystem and network effects matter more.
Did everyone forget GoogleTV? Amazon has a strong presence in the retail market and my amazon kindle dx is still supported. I'm not wasting my time with google again.
One critical difference: GoogleTV was a platform where distribution (updates, etc.) were owned by the TV manufacturer. Google owns the update channel on Google Home.
It's like comparing the Pixel now to early carrier-operated Android phones.
