BMW’s Alexa integration gets it right

BMW will in a few days start rolling out to many of its drivers support for Amazon’s Alexa voice assistant. The fact that BWM is doing this doesn’t come as a surprise, given that it has long talked about its plans to bring Alexa — and potentially other personal assistants like Cortana and the Google Assistant — to its cars. Ahead of its official launch in Germany, Austria, the U.S. and U.K. (with other countries following at a later date), I went to Munich to take a look at what using Alexa in a BMW is all about.

As Dieter May, BMW’s senior VP for digital products told me earlier this year, the company has long held that in-car digital assistants have to be more than just an “Echo Dot in a cup holder,” meaning that they have to be deeply integrated into the experience and the rest of the technology in the car. And that’s exactly what BMW has done here — and it has done it really well.

What maybe surprised me the most was that we’re not just talking about the voice interface here. BMW is working directly with the Alexa team at Amazon to also integrate visual responses from Alexa. Using the tablet-like display you find above the center console of most new BMWs, the service doesn’t just read out the answer but also shows additional facts or graphs when warranted. That means Alexa in a BMW is a lot more like using an Echo Show than a Dot (though you’re obviously not going to be able to watch any videos on it).

In the demo I saw, in a 2015 BMW X5 that was specifically rigged to run Alexa ahead of the launch, the display would activate when you ask for weather information, for example, or for queries that returned information from a Wikipedia post.

What’s cool here is that the BMW team styled these responses using the same design language that also governs the company’s other in-car products. So if you see the weather forecast from Alexa, that’ll look exactly like the weather forecast from BMW’s own Connected Drive system. The only difference is the “Alexa” name at the top-left of the screen.

All of this sounds easy, but I’m sure it took a good bit of negotiation with Amazon to build a system like this, especially because there’s an important second part to this integration that’s quite unique. The queries, which you start by pushing the usual “talk” button in the car (in newer models, the Alexa wake word feature will also work), are first sent to BMW’s servers before they go to Amazon. BMW wants to keep control over the data and ensure its users’ privacy, so it added this proxy in the middle. That means there’s a bit of an extra lag in getting responses from Amazon, but the team is working hard on reducing this, and for many of the queries we tried during my demo, it was already negligible.

As the team told me, the first thing it had to build was a way to switch that can route your queries to the right service. The car, after all, already has a built-in speech recognition service that lets you set directions in the navigation system, for example. Now, it has to recognize that the speaker said “Alexa” at the beginning of the query, then route it to the Alexa service. The team also stressed that we’re talking about a very deep integration here. “We’re not just streaming everything through your smartphone or using some plug-and-play solution,” a BMW spokesperson noted.

“You get what you’d expect from BMW, a deep integration, and to do that, we use the technology we already have in the car, especially the built-in SIM card.”

One of the advantages of Alexa’s open ecosystem is its skills. Not every skill makes sense in the context of the car, and some could be outright distracting, so the team is curating a list of skills that you’ll be able to use in the car.

It’s no secret that BMW is also working with Microsoft (and many of its cloud services run on Azure). BMW argues that Alexa and Cortana have different strengths, though, with Cortana being about productivity and a connection to Office 365, for example. It’s easy to imagine a future where you could call up both Alexa and Cortana from your car — and that’s surely why BMW built its own system for routing voice commands and why it wants to have control over this process.

BMW tells me that it’ll look at how users will use the new service and tune it accordingly. Because a lot of the functionality runs in the cloud, updates are obviously easy and the team can rapidly release new features — just like any other software company.

Why BMW needs to own its customer experience from start to finish

For the last few years now, BMW has wrestled with the question of what it’ll mean to be a luxury car manufacturer in the age of electric cars, autonomous driving and rapidly changing — and increasing — customer expectations. What, after all, makes something the “ultimate driving machine” when the driver eventually stops driving?

For BMW, the answer is a renewed focus on technology and the in-car experience it enables, without forgetting its heritage in performance cars. To discuss the state of the company’s transformation, not just in terms of its cars but also its business model, I sat down with BMW’s outspoken VP of Digital Products and Services Dieter May shortly after the company unveiled the latest version of its in-car operating system.

“We build digital products and services that are meant to help us differentiate our core product, the car and generate revenue,” May said. “But these digital services also provide us with channels and touch points that allow us to now have a direct relationship with the customer on the sales side and talk to the customer directly.”

In the car industry, however, the sales channel has traditionally been the dealership. That’s where you buy the car and that’s where you get it serviced. It’s the dealer who knows (ideally) who you are and what you want. The manufacturer’s role in this model is to build the car, maybe build a bit of a central online presence with a configurator so customers can get some idea of the car’s price — and get out of the way.

That’s not the future that May envisions, though. And neither is it one where the big tech companies like Apple and Google own the driver and the user experience.

“As we’re building the digital products in the car, we are also building out the car as a channel and touchpoint at the same time,” May noted. “We’ll have our app, a personal assistant etc. and with that, we can create a user profile and provide that to our sales teams. Today, virtually every car manufacturer can’t talk directly to the customers because the customer belongs to the dealer, and because the different business units, like after sales, financial services, etc., aren’t unified and all try to talk to the customer separately.”

So for BMW, digital experiences in the car are one thing — and you can expect to hear a bit more about this in the coming weeks and months — but the company is looking beyond this and how it can use this transformation to also create new business opportunities that go beyond maybe selling an in-car Spotify subscription for a few dollars. But what gets May most excited about this is the prospect of being able to talk to the customer throughout the ownership lifecycle. “That’s the cool part, because it allows me to keep the product ‘car’ fresh throughout the lifecycle and manage it like a device,” he said.

The move that May is hinting at here means that BMW wants to not just focus on selling cars but to create a model where it can extract some revenue from users throughout the car’s life. As an example, May noted that BMW may sell you a Mini with a charging package and, in addition, it’ll sell you a flat-rate subscription to charge it. “There are so many opportunities here, but you have to play it smart, both before somebody buys the car and after the sale.

If BMW wants to own the customer, though, that means dealerships have to change. “The dealers will have to grow into a different role over time,” May acknowledged. “We expect and hope that just as we will share data with the dealer, the dealers will share their data with us. A small piece of a larger cake is still better than nothing.”

Customers will still come to the dealer for their service needs, so BMW isn’t cutting them out completely, but the company definitely wants to own a larger part of the relationship with the customer. And at the end of the day, it’s the dealer who represents the manufacturer, whether that’s taking somebody on a test drive or helping the customer take delivery of a car.

“What’s most important for us — and everybody is talking about autonomous driving and electric vehicles and so on — but if we don’t become a customer-centric company, then we are destined to fail. The number of digital elements in our customers’ lives and in the car continues to increase, and if we don’t understand that, we’ve got a problem.”

As for its current in-car systems, May told me that BMW now has more than three million registered users for its ConnectedDrive system, but what’s maybe more important is that the number of user interactions is increasing significantly faster than that. What’s interesting to hear is that the way BMW thinks about these users is pretty much in line with any consumer internet company. The team tracks monthly, weekly and daily active users, for example, and is working to increase those engagement numbers with every update.

One problem car manufacturers have long suffered from is that cars stick around far longer than smartphones, and that the in-car technology can quickly seem out of date. Because it is betting on a connected car that is always connected to the cloud, BMW (and, to be fair, many of its competitors) is now able to update the in-car software. That’s true for new head units, but not necessarily for older ones, and fragmentation remains an issue — though with a standardized model for both BMW and its Mini brand, that’ll likely be less of a problem for newer cars than for those that launched two or three years ago.

“In the car industry, a lot of people think that everything has to be backward-compatible reaching back 20 years, but my take is that we have to be more like smartphone vendors,” said May.

Taking a page from the software industry, the BMW team often launches new features that are akin to minimal viable products. That’s not necessarily something the luxury car buyer is used to, of course, but it does allow the company to test new features and expand on them as they gain traction.

The next concrete step for BMW in this journey is to feature an interactive personal assistant in the car that knows about the customer. May believes this will drive a lot of usage. Although the exact details remain to be seen, the BMW team hinted that we’ll learn more in the fall.

With its new in-car operating system, BMW slowly breaks with tradition

When you spend time with a lot of BMW folks, as I did during a trip to Germany earlier this month, you’ll regularly hear the word “heritage.” Maybe that’s no surprise, given that the company is now well over 100 years old. But in a time of rapid transformation that’s hitting every car manufacturer, engineers and designers have to strike a balance between honoring that history and looking forward. With the latest version of its BMW OS in-car operating system and its accompanying design language, BMW is breaking with some traditions to allow it to look into the future while also sticking to its core principles.

If you’ve driven a recent luxury car, then the instrument cluster in front of you was likely one large screen. But at least in even the most recent BMWs, you’ll still see the standard round gauges that have adorned cars since their invention. That’s what drivers expect and that’s what the company gave them, down to the point where it essentially glued a few plastic strips on the large screen that now makes up the dashboard to give drivers an even more traditional view of their Autobahn speeds.

With BMW OS 7.0, which I got some hands-on time with in the latest BMW 8-series model that’s making its official debut today (and where the OS update will also make its first appearance), the company stops pretending that the screen is a standard set of gauges. Sure, some of the colors remain the same, but users looking for the classic look of a BMW cockpit are in for a surprise.

“We first broke up the classic round instruments back in 2015 so we could add more digital content to the middle, including advanced driving assistance systems,” one of BMW’s designers told me. “And that was the first break [with tradition]. Now in 2018, we looked at the interior and exterior design of our cars — and took all of those forms — and integrated them into the digital user interface of our cars.”

The overall idea behind the design is to highlight relevant information when it’s needed but to let it fade back when it’s not, allowing the driver to focus on the task at hand (which, at least for the next few years, is mostly driving).

So when you enter the car, you’ll get the standard BMW welcome screen, which is now integrated with your digital BMW Connected profile in the cloud. When you start driving, the new design comes to life, with all of the critical information you need for driving on the left side of the dashboard, as well as data about the state of your driving assistance systems. That’s a set of digital gauges that remains on the screen at all times. On the right side of the screen, though, you’ll see all of the widgets that can be personalized. There are six of those, and they range from G meters for when you’re at a track day to a music player that uses the space to show album art.

The middle of the screen focuses on navigation. But as the BMW team told me, the idea here isn’t to just copy the map that’s traditionally on the tablet-like screen in the middle of the dashboard. What you’ll see here is a stripped-down map view that only shows you the navigational data you need at any given time.

And because the digital user interface isn’t meant to be a copy of its analog counterpart from yesteryear, the team also decided that it could play with more colors. That means that as you move from sport to eco mode, for example, the UI’s primary color changes from red to blue.

The instrument cluster is only part of the company’s redesign. It also took a look at what it calls the “Control Display” in the center console. That’s traditionally where the company has displayed everything from your music player to its built-in GPS maps (and Apple CarPlay, if that’s your thing). Here, BMW has simplified the menu structure by making it much flatter and also made some tweaks to the overall design. What you’ll see is that it also went for a design language here that’s still occasionally playful but that does away with many of the 3D effects, and instead opted for something that’s more akin to Google’s Material Design or Microsoft’s Fluent Design System. This is a subtle change, but the team told me that it very deliberately tried to go with a more modern and flatter look.

This display now also offers more tools for personalization, with the ability to change the layout to show more widgets, if the driver doesn’t mind a more cluttered display, for example.

Thanks to its integration with BMW Connect, the company’s cloud-based tools and services for saving and syncing data, managing in-car apps and more, the updated operating system also lays the foundation for the company’s upcoming e-commerce play. Dieter May, BMW’s VP for digital products and services, has talked about this quite a bit in the past, and the updated software and fully digital cockpit is what will enable the company’s next moves in this direction. Because the new operating system puts a new emphasis on the user’s digital account, which is encoded in your key fob, the car becomes part of the overall BMW ecosystem, which includes other mobility services like ReachNow, for example (though you obviously don’t need to have a BMW Connect account just to drive the car).

Unsurprisingly, the new operating system will launch with a couple of the company’s more high-end vehicles like the 8-series car that is launching today, but it will slowly trickle down to other models, as well.

Here are the experts who will help shape Europe’s AI policy

The European Commission has announced the names of 52 experts from across industry, business and civil society who it has appointed to a new High Level Group on AI which will feed its strategy and policymaking around artificial intelligence.

In April the EU’s executive body outlined its approach to AI technology, setting out measures intended to increase public and private investment; prepare for socio-economic changes; and ensure an appropriate ethical and legal framework.

The High Level Group is a key part of the Commission’s AI strategy as the experts will feed its policymaking here by making detailed recommendations on ethical, legal and societal issues.

The EC put out a call for experts for this “broad multi-stakeholder forum” back in March.

The group announced today is comprised of 30 men and 22 women, and includes industry representatives from AXA, Bayer, Bosch, BMW, Element AI, Google, IBM, Nokia Bell Labs, Orange, Santander, SAP, Sigfox, STMicroelectronics, Telenor and Zalando.

Google is represented by Jakob Uszkoreit, an AI Researcher in the Google Brain team.

Also in the group: Jaan Tallinn, a founding engineer of Skype and Kazaa, and a former investor in and director of the Google-acquired AI company DeepMind.

European civil society bodies represented in the forum include consumer rights group BEUC; digital rights group Access Now; algorithmic transparency advocacy group AlgorithmWatch; the EESC civil society association; the ETUC which advocates for workers rights and well being; and Austrian association that supports the blind and visually impaired.

The list also includes representatives from several technology associations, along with political advisers and policy wonks, and academics and legal experts of various stripes.

The full list is here.

Towards a comprehensive AI strategy

Back in April the Commission said it hoped to be able to announce a “coordinated plan on AI” by the end of the year — after saying, in March, that a “comprehensive European strategy on AI” was on the way “in the coming months”.

“As any technology that has a direct impact on people’s lives and work, the emergence of AI also raises legitimate concerns that should be addressed to build trust and raise awareness,” it wrote then. “Given the broad impact AI is expected to have, the full participation of all actors including businesses, academics, policy makers, consumer organisations, trade unions, and other representatives of the civil society is essential.”

The multi-stakeholder forum is also intended to serve as the steering group for the work of another, even broader multi-stakeholder forum — also announced in April, and called the European AI Alliance — which the Commission said will include an online platform to allow for anyone who wants to participate to sign up and join in the discussion.

So the High Level Group is basically an AI expert talking shop intended to support this more public AI talking shop — to try to achieve some kind of pan-EU consensus on how to respond to the myriad socio-economic and ethical challenges that flow from the increasingly use and capabilities of autonomous technologies.

In terms of specific tasks for the group, the Commission says it will be tasked to:

  • advise it on next steps addressing “AI-related mid to long-term challenges and opportunities”, feeding policy development, legislative evaluation and next-gen digital strategy;
  • propose draft AI ethics guidelines — covering issues such as “fairness, safety, transparency, the future of work, democracy and more broadly the impact on the application of the Charter of Fundamental Rights, including privacy and personal data protection, dignity, consumer protection and non-discrimination”;
  • and help with “further engagement and outreach mechanisms to interact with a broader set of stakeholders in the context of the AI Alliance, share information and gather their input on the group’s and the Commission’s work”

Also today, the London-based Chatham House international policy think tank has published a report looking at the short to medium-term policy challenges posed by AI, focusing on military, human security and economic perspectives.

The report warns generally of the need for a framework for better managing the rise of AI to ensure it does not simply serve to reinforce existing inequalities.

Among its specific recommendations are: That clear codes of practice be developed for policymakers and states to utilize AI for decision-making purposes; that funding be allocated for deploying and developing AI systems with humanitarian goals; better education and training for policymakers and technical experts in each other’s respective domains; that governments to invest in developing and retaining homegrown AI talent and expertise to “become independent of the dominant AI expertise now typically concentrated in the US and China”; and that strong relationships be developed between public and private AI developers to ensure innovation driven by the commercial sector permeates to the use of AI in the public sector.

Hear from the executives of Innoviz and Oryx Vision about the eyes and ears of the new automobile in Tel Aviv

The success of the autonomous vehicle revolution relies on complicated systems of sophisticated sensors working in harmony to provide the magic of sight to machines.

OmerKeilaf, chief executive, Innoviz

In Tel Aviv, we’ll hear from experts in the field as they discuss the technological marvels that are the driving force behind the transformation of mobility in the modern world.

Omer David Keilaf, the chief executive of Innoviz, comes to us with some significant recent wins under his company’s belt. The Innoviz LIDAR technology has been selected by BMW to power its Level 3 to Level 5 autonomous vehicle systems.

The company’s solid-state LiDAR sensor, available as a built-in device beginning next year, is much smaller than traditional LIDAR and is stationary.

Before founding Innoviz, Keilaf led the system and product definition efforts at the world’s first handheld molecular sensor for mobile devices with ConsumerPhysics. Previous roles include leading the system architecture and engineering teams at bTendo (acquired by ST Micro) and Anobit (acquired by Apple) .

Rani Wellingstein, chief executive, Oryx Vision

No less impressive is the work of fellow panelist Rani Wellingstein of Oryx Vision, whose company is developing its own novel LIDAR technology. Oryx’s LiDAR uses antennas in place of photodetectors to retrieve both range and velocity information for the points of light in its high-resolution scans of its surroundings. The company claims that its technology is a million times more sensitive than existing LiDAR systems, and is better able to deal with interference from sunlight, and from other LiDARs in operation on the road.

A serial entrepreneur, Wellingstein’s last company, Intucell, was sold to Cisco for $475 million in 2013. At Cisco, Wellingstein served as the vice president and business unit manager of Cisco’s self optimized networks business unit.

 

Israel is driving autonomous innovation and we’re excited to talk to the folks behind the wheel of the nation’s innovative companies. Join us. You have just 48 hours left to score the early-bird ticket price — 265 ILS. So buy your tickets now.