Emmanuel Maggiori of Artificial Intelligence: The Soul-Draining Illusion That Plagues Corporate AI Projects And Demands A Return To Tangible Engineering

As part of our series about the future of Artificial Intelligence, I had the pleasure of interviewing Emmanuel Maggiori.

Emmanuel Maggiori, PhD, is a freelance software engineer specialized in machine learning and scientific computing. He has developed AI for a wide variety of applications, from extracting objects from satellite images to packaging holiday deals for millions of travelers every day. He’s the author of Smart Until It’s Dumb: Why artificial intelligence keeps making epic mistakes (and why the AI bubble will burst).

Thank you so much for joining us in this interview series! Can you share with us the ‘backstory” of how you decided to pursue this career path in AI?

I wouldn’t say I chose to pursue a career in AI; it landed on me! I was doing an internship at a research lab in France and they offered me a contract to stay there and do a PhD in AI. They gave me two days to decide whether I wanted to take the opportunity and I said, “Yes, why not?”

After finishing the PhD, I wanted to build more tangible products, so I left academia and applied for positions in data science, a job that usually involves using AI to attain business goals. I worked in the travel field for a few years. For example, at Expedia, I wrote AI-powered algorithms that helped put together travel packages for millions of travelers every day. It was crazy looking at the dashboard and seeing that, every single minute, thousands of people from all over the world were using what we’d built. It was high-impact and really rewarding work. However, I wanted to do something more independent, so I decided to strike out on my own and have worked as a freelancer since then.

Unfortunately, my experience working in the AI field has been a bit bumpy. I soon noticed that a lot of companies want to use AI just because everyone else does, sometimes not even really knowing what they’ll use it for — they put the cart before the horse. This leads to projects that fail in a predictable way. The thing is, few people understand how AI works. So, when a team notices that what they’re building doesn’t make much sense, they often keep pretending it does instead of telling the truth to the decision-makers in the organization. As a result, bogus AI projects can drag on for a long time without anyone else realizing it.

One of my latest experiences of that kind happened when I worked for a huge multi-national company. They had built an internal AI product to help its executives make important business decisions, and I joined the project halfway through to improve the existing product. I soon realized that the tool didn’t work as intended and that it would be very hard to fix it, as we didn’t really have the necessary data to build the required AI system. So, instead of telling the truth, our managers made us add a layer that overrode the AI’s outputs to make them look more convincing. In fact, we had to make the outputs match the decisions that executives made. This way the executives had an AI tool to justify their decisions though this wasn’t coincidental. At one point, we had tampered with the outputs of this system so much that the underlying AI wasn’t used anymore — we just pretended to be using it. The whole thing was soul draining.

As this kind of thing happened not once but many times, I came to the realization that AI can be amazing but, behind every successful project, there are many projects with less flattering outcomes that are swept under the rug.

That experience pushed me into a career break — I took some time off to do some thinking. During that time, I wrote my book Smart Until It’s Dumb, which tells the story of AI and why it’s so powerful, but it also tells the story of the immense hype and questionable practices that plague the field.

Since then, I’ve decided to step away from the field, at least a little bit. I’m helping companies build challenging software, whether it involves AI or not. After all, I’m an engineer and love to build products that people use. Sometimes AI is necessary for that, sometimes it’s not.

What lessons can others learn from your story?

My first suggestion is: Don’t fall for the hype. And not just the AI hype — be cautious about blockchain, big data, ChatGPT, metaverse or any of those buzzwords that people keep using all the time. As an engineer, your job is to find the best solution to a problem. While you may specialize in a certain technique, you should avoid pushing it everywhere; sometimes it’s just not the best tool.

If you have the privilege of choosing who to work for, try to join a company that is building a real, tangible product that adds or will add true value to clients. And do your due diligence to make sure it’s at least somehow feasible to build it.

Can you tell our readers about the most interesting projects you are working on now?

As part of my freelance work, I’m working with an early-stage start-up to build a product that helps save energy in large institutional buildings. We’re trialing the software in one of the UK’s most important universities. The software is connected to smart radiator valves in two university buildings and makes adjustments to the valves remotely every few minutes to try to reduce energy consumption. It’s really cool. I built the product end to end, from the nerdy part (mathematical optimization, modeling of temperature dissipation, etc.) to the back end of the software — which lives behind the scenes — to the user interface which lets building managers browse floor plans to monitor room temperature and energy costs. This is an all-round project that is as real and exciting as it can get.

None of us are able to achieve success without some help along the way. Is there a particular person who you are grateful towards who helped get you to where you are? Can you share a story about that?

One of my good friends from high school. We’re polar opposites in many ways, and that’s precisely why he’s been so inspirational and has helped me step out of my comfort zone. I’m quite risk averse; he, on the contrary, has traveled the world (at one point he played the violin on the street to make money) and now owns a successful niche business that sells products to really happy customers.

What are the 5 things that most excite you about the AI industry? Why?

• The use of applied mathematics. I’m quite a nerd and have always loved math. But I’m aware that, for a lot of people, math appears dull and useless. Well, AI is full of math. In fact, it’s a concrete application of math that we all interact with every day.
• The tools. When I started working in this field, getting your AI algorithm to work was quite cumbersome. Now a set of really mature tools have been developed (like the Pandas and TensorFlow software libraries) which make working in AI much easier without getting lost in technical details.
• The job opportunities. When I post my CV on a job board, I tend to get many calls from recruiters right away. I’m not sure how long this will last though, especially considering the recent dramatic layoffs in the tech sector.
• The variety. It’s quite common for AI practitioners to switch industries completely, from travel to energy, or from banking to media. The sky is the limit.
• The melting pot. I’ve met people in this field from everywhere in the world who changed careers from civil engineering, physics, accounting… You name it!

What are the 5 things that concern you about the AI industry? Why?

• The hype. In 2014, Ray Kurzweil predicted that artificial general intelligence, meaning computers with human-level performance at every task, will be attained by 2029. In 2022, Elon Musk agreed with that prediction, saying he would be surprised if it doesn’t happen by 2029. But this isn’t the first time; that prediction is as old as computing itself. In 1960, for example, economist Herbert A. Simon said, “Machines will be capable, within twenty years, of doing any work that a man can do.” We are in the middle of the third wave of hype around AI (the previous two, in the 60s and 80s, ended in disappointment). People say, “But this time it’s for real,” although they said the same the previous two times. I think it would be great if we were more realistic about AI because the hype pushes people to try it when they don’t need it or aren’t ready for it.
• Our safety. I don’t think the current AI methodology — dominated by machine learning — can be used safely in life-critical systems like self-driving cars. The proof is in the pudding: despite repeated attempts and 200 billion in investment, the industry of self-driving cars hasn’t been able to produce fully autonomous self-driving cars yet (that is, autonomous cars which can drive themselves anywhere, mixed with pedestrians, cyclists and other non-autonomous traffic). Stories of accidents and silly mistakes abound, such as a Waymo car getting severely confused by a row of traffic cones. I am genuinely scared that people will get caught up in the hype, thinking that AI is infallible, and thus try to deploy unsafe AI.
• The illusory patterns. Machine learning, the technique behind current AI, tries to find patterns in data and learn from them. But, sometimes, it finds patterns in the data that don’t make sense — they are spurious correlations. This sometimes makes you believe that AI works well when developing it, only to be negatively surprised when the system goes live. In my experience, when you build traditional software, a tiny bug tends to get overblown (you see a blue screen or a frozen webpage). In machine learning, on the contrary, tiny bugs tend to make you believe that AI works better than it actually does. You have to be extra careful.
• The lies. Unfortunately, I’ve seen many people play it fast and loose with the truth when it comes to describing and promoting the performance of their AI products. Since few people understand AI outside the field, it is particularly easy to exaggerate AI’s benefits without consequences. I’ve seen companies waste millions pursuing projects that were doomed to fail from the start yet no one gave the decision-makers a heads-up.
• The fanaticism. Within the AI community, there are many hardcore fanatics that refuse to accept that maybe machine learning isn’t enough to create truly intelligent machines. When AI makes mistakes, they blame it on a scapegoat, such as, “the training data wasn’t varied enough,” instead of analyzing the overall approach more critically.

As you know, there is an ongoing debate between prominent scientists, (personified as a debate between Elon Musk and Mark Zuckerberg,) about whether advanced AI has the future potential to pose a danger to humanity. What is your position about this?

I think we probably should not be afraid in the way a lot of people seem to be afraid, worrying about robots gone rogue, machine-led mass extinction, skyrocketing unemployment, etc. But we might need to be concerned about the consequences of humans being overly optimistic and putting too much trust in the capabilities of AI. Current AI makes mistakes that humans would never make, like confusing a toothbrush with a baseball bat. And it can also be easily fooled. For example, we can alter traffic signs in ways that are imperceptible to the human eye but make the computer misunderstand them. Current AI also struggles in new or uncommon situations that were not present in the data it learned from, but the ability to deal with unprecedented situations is paramount for safety. So, the real fear with AI is that overly optimistic people will think it is infallible and try to deploy it in unsafe ways.

What can be done to prevent such concerns from materializing? And what can be done to assure the public that there is nothing to be concerned about?

I think it would be great if the public understood better how current AI works. While becoming an AI practitioner requires learning quite a lot of things, the general principles of machine learning can be easily understood.

Most of current AI scans a database of examples of how to do the job and tries to learn useful rules to imitate it. For instance, to train language models like ChatGPT, lots of sentences are collected from the internet and the last word is removed from them. For instance, we turn the sentence “Her Majesty Queen Elizabeth” into “Her Majesty Queen…” and we tell the system that the corresponding next word for that interrupted sentence is “Elizabeth.” And we train the system to learn such rules to guess the next word in a wide variety of sentences. Also, for ChatGPT, instead of just using sentences collected from the internet, people were hired to manually write down tens of thousands of examples especially designed to improve ChatGPT and make it better at dialogue.

So, while ChatGPT is a great imitator — and can certainly put together really credible sentences — it lacks a high-level internal model of the world. If you ask it a strange question that is unlikely to be learned just from looking at sentences from the internet, it fails. Have a look at an example. When I asked ChatGPT whether an anaconda fits in a shopping mall, its answer was:

“It is not possible for an anaconda to fit inside a shopping mall, as anacondas are typically much larger than the size of a shopping mall. Anacondas are some of the largest snakes in the world, and can grow to be over 20 feet long and weigh over 100 pounds. A shopping mall, on the other hand, is a large indoor commercial building that typically has multiple floors and is designed for retail shopping and other activities. An anaconda would not be able to fit inside a shopping mall, even if it was empty.”

As you can see, ChatGPT is often confidently wrong. People need to understand this and why it happens so they aren’t easily fooled when others promote the use of AI everywhere (it even appears that a Colombian judge used ChatGPT for a court ruling). So, let’s understand AI — its power but also its limitations — and let’s not deploy it carelessly in systems that could endanger us.

How have you used your success to bring goodness to the world? Can you share a story?

Over the past few years, people have been reaching out to me asking for help to get a job in data science. When that happens, I first ask them to show me their CVs. In almost every case, the CVs look subpar (just like my first CV) and that’s the reason why they aren’t getting any calls. A friend of mine, who had been struggling to get calls from recruiters asked me for help. When I looked at his CV I found out it was filled with acronyms internal to his latest company that only his coworkers would understand (“I worked with the TMs on the UP project”). Recruiters only spend seven seconds looking at a CV before discarding it, so the message needs to be crystal clear. I helped him revamp his CV and he got a job soon after.

Since I noticed this pattern of poorly written CVs many times, I decided to put together a free mini-course on Udemy (“How to get a data science job without experience”) to share my advice with others. It’s been watched by over 2,500 students and many have reached out to tell me they’ve found it helpful. I had a couple of phone calls with one of the students to help him rewrite his CV, and he went from zero calls from recruiters to landing his first data science job a couple of weeks later. I’ve found all of this very rewarding and I’m glad it’s helped other people attain their career goals.

As you know, there are not that many women in your industry. Can you advise what is needed to engage more women into the AI industry?

My first thought is to say that, if you’re a woman and want to work in the AI field, don’t let anything or anyone stop you! I’ve met many women with remarkably successful careers in the AI industry. My PhD supervisor, for instance, had worked for NASA and the French IT research lab before she was 30, and she now directs a research team in the private sector.

What is your favorite “Life Lesson Quote”? Can you share a story of how that had relevance to your own life?

“Trust your gut.” As techies, we tend to be a little too mathematical when making decisions, weighing pros and cons or trying to do what’s “objectively” right. However, quite often your gut speaks; it tells you that a project or job offer “doesn’t feel right” even though it looks good on paper. My experience tells me that gut feelings tend to be right. One such experience was when I accepted to do my PhD. My gut was telling me it wasn’t for me; I wanted to work on something more practical and real. However, everyone kept telling me that doing a PhD was the right decision and it was an opportunity I shouldn’t miss. As I did the PhD, I noticed over and over that it wasn’t the right thing for me as I’d suspected all along. I still finished it, but sometimes I wish I’d trusted my gut.

You are a person of great influence. If you could start a movement that would bring the most amount of good to the most amount of people, what would that be? You never know what your idea can trigger. :-)

I would love to start an academy to train people in software engineering and data science, especially students who don’t want to follow the traditional path of attending university. It seems to me that traditional university is pretty broken; students spend a lot of time learning useless or outdated stuff and they do not learn some important, practical things. I was myself surprised at how many things I had to learn on the job that I wish someone had taught me before. I feel students are often “paying their dues” just to get a degree rather than being properly trained. Since software development is not a regulated profession, you don’t really need a degree to get a job. It can help, of course, but it’s not a hard requirement. So, I’d love to train people to become successful software engineers without going through the fuss of traditional education.

How can our readers further follow your work online?

Just follow me on LinkedIn (https://linkedin.com/in/emaggiori) or check out my homepage (https://emaggiori.com).

Thank you so much for joining us. This was very inspirational, and we wish you continued success in your important work.

About The Interviewer: David Leichner is a veteran of the Israeli high-tech industry with significant experience in the areas of cyber and security, enterprise software and communications. At Cybellum, a leading provider of Product Security Lifecycle Management, David is responsible for creating and executing the marketing strategy and managing the global marketing team that forms the foundation for Cybellum’s product and market penetration. Prior to Cybellum, David was CMO at SQream and VP Sales and Marketing at endpoint protection vendor, Cynet. David is a member of the Board of Trustees of the Jerusalem Technology College. He holds a BA in Information Systems Management and an MBA in International Business from the City University of New York.