5G Security: The digital ‘Beware of Dog’ sign
Podcast episode 32
Varonis’ Field CTO and hacker Brian Vecci spent his life breaking into computers. With the roll-out of 5G, the number of break-in opportunities is set to soar. And if you think the biggest security risk is the Post-It Note password, you’d be wrong.
Below is a transcript of this conversation. Some parts have been edited for clarity.
Michael Hainsworth: Brian Vecci of Varonis has spent his life hacking into systems and with the rollout of 5G, the number of systems is set to soar. As telecom companies evolve from dumb pipes to communications service providers, I asked this professional digital break in artist, what is it the industry needs to address as the technology is deployed? Particularly, since CSPs do not have a history of running server firms like Amazon, Google and the rest do. So I begin our conversation by discussing the role CSPs have in security for enterprise and residential customers.
Brian Vecci: Telecoms, from a security perspective, definitely have a responsibility on both sides. From a consumer perspective, telecoms often are the end point. For a lot of telecom providers, they are providing the network endpoint that their consumers are using. So the answer then is of course not, because if Verizon is providing me a router that is now my home wifi network, which in a remote work environment, which we're all in now because of the pandemic, is now also an end point network for my company. Verizon absolutely has... I don't want to pick on Verizon; they just happened to be my telecom provider. They have a responsibility to make sure that endpoint is secure and is logged, and isn't a weak point for both their networks and my company's corporate network.
From a business perspective, telecoms are really providing much more of a service in the businesses. The onus is on them to make sure that their endpoints, wherever they connect to the telecom networks, are as secure as possible. The short answer to your question is no, or kind of, “Yes, and…” if that makes some sense.
MH: Well, as telecom companies continue to evolve from dumb pipes to communications service providers, and the consumers evolve from desktop computers to mobile, which areas are most interesting for hackers? Is it consumer devices like phones, consoles, and PCs? Is it IoT devices? Is it the telco Networks or the IT backend systems, like customer databases?
BV: The answer is, it depends on the kind of attacker you're talking about and what their targets are. There are different kinds of hackers. There's not one single club of attackers that are sitting around saying, here's what we wanna go after.
MH: You mean, there isn't some teenager with a hoodied, hunched over a computer in the dark?
BV: So they're not in the dark by the way. And there absolutely is a teenager in a hoodie somewhere. But there are also very well organized hacking groups like Maze that are really advanced persistent threats that often have better customer service than the security vendors (which is a whole separate discussion) that are going big game hunting, and going after the really big targets.
What's interesting that you asked about, the teenager and the hoodie. The teenager in the hoodie, if he gets access to really sensitive sets of intellectual property inside a government contractor, for instance, teenager in the hoodie, isn't gonna know what the heck to do with that stuff. Whereas an advanced, persistent threat, either a nation state, or a for-profit hacking group, absolutely is. Probably the teenager in the hoodie, to use this in metaphor, is probably really interested in... You said something really interesting about, consumer devices moving from desktops to mobile, to IoT.
What everybody has to remember about IoT is that, what you're talking about are internet connected computers that serve a primary purpose, but I'm looking in my apartment right now at a Sonos speaker. I love this thing. But it is an internet connected computer. It's compute, it's got an interface, it's got authentication, it's got authorization, ‘cause I can have different users that can connect to it. And it is connected to the wider internet, of course, through my home network. It is basically, and probably as powerful and as capable as some of the desktops that I had 15 and 20 years ago. So the move from desktops to mobiles, to IoT. Another way to think about that is that every single person now, every single user, every single consumer now doesn't just have one compute device, they have multiple compute devices. And in some cases, dozens of compute devices in a single home. Your Ring security system, your Sonos speakers, your home security system that might tie everything together inside your home, any home automation. Not to mention, the number of personal compute devices that you're using, whether it's a corporate laptop, your personal laptop, a tablet, a phone, your game consoles... Everything is an internet connected computer today. Even the guitar amplifier that I have now, is an internet connected computer. And so all of those, as a really long introduction, are exceptionally valuable attack vectors for someone who's looking for the easiest way into a network to either steal resources, financial credentials, or lock down data in the case of Ransomware.
MH: But that's that path of least resistance that you talk about. It strikes me that the advanced persistent threats that you suggest, nation states or organizations that are dedicated to hacking, are the greater threat as we make this evolution. What are they targeting? Is it the consumer phone? Is it the corporate IoT? Is it the telco network? What's the most interesting? What makes them salivate?
It was clear, this was an organized development team that had a release cycle, that was doing QA and then pushing updates to their malware in the wild.
BV: Everything. And we've seen that. So one of the things that Varonis does, is we monitor... I mean, we're not a consumer focused security platform. We work with businesses, organizations that are trying to secure their enterprises, especially in a bubble in a remote work environment.
The answer is, and this is what we see over and over and over again, that the really powerful hacking groups are using every tool in their bag. They'll hack your phone, they'll go hack a Sonos speaker. They will hack and open a Kerberos password that hasn't been reset in a couple of years. They will use every exploit that they can possibly use. What we see over and over again is that the advanced groups are just gonna go through every technique that they possibly can. And what they have the resources to do, is take all of these exploits and techniques and... this is a really scary word, I was going to say weaponize, productized is maybe a better way of thinking about it.
What you can do is you can take 20 different exploits, put them together in a set of scripts, and then make it really easy to use all of them. If the teenager in the hoodie doesn't have that kind of resources, doesn't have that kind of development time... One of the things that we saw, we caught an attack last year, and it was a media company that had ransomware and malware all over their network. And the really interesting novel attack that we saw was a variant of malware that nobody had ever seen before. As we were working with this company, trying to eradicate it, figure out how it was spreading, make sure that they had the proper monitoring in place (which we will get to... always have an audit trail), we were seeing updates to the ransomware coming down from different locations in the world, in sometimes different languages.
It was clear, this was an organized development team, that had a release cycle, that was doing QA and then pushing updates to their malware in the wild. These are really sophisticated development organizations, and they will productize the attacking both the techniques, the technology to do it, the methods used, and they will keep things up to date to make sure that they have the most efficient and effective malware out there.
MH: Tell me then of those key areas, which are the least protected? My assumption is IoT, because that's the one that gets all of the headlines. Is the Internet of Things component to Industry 4.0 the entry point for that advanced persistent threat?
BV: Maybe. It's probably the most fun to talk about because it's emblematic of the change that we have seen and continue to see. Like there wasn't an Internet of Things five, ten years ago. So it's really interesting to talk about.
The more boring thing to talk about is the blocking and tackling, to use a football metaphor, the basics of security. And here's what I mean. At Varonis, we do risk assessments of organizations of all sizes. Everywhere from law firms that are a couple hundred people, up to the biggest banks and insurance companies in the world with hundreds of thousands of users and petabytes of data. You know the single biggest problem that every single company faces? And it sounds so boring because it was a problem that I was trying to solve while I worked on a help desk 20 years ago.
MH: Let me guess, the Post-It note with the password on it.
BV: No! That isn't even that big of a problem because you know why? Your advanced persistent threat or your hacker in the hoodie, you know what he can't do? He can't read a posted note stuck on the side of your monitor.
MH: And that's where social engineering comes in?
Every company faces this problem, that way too much of their data is wide open.
BV: Yeah. But he's not gonna get on a plane and fly to London to do that. Right? He's just not. The Post-It note on the side of the monitor, it's fun to make fun of, but it's a lot better security than having the exact same password that you used for some message board or some nonsense app that you downloaded, and that password is also the same the password that you use for your VPN or Office 365 account.
The problem is, if an attacker gets access to credit and Valid credentials, that's the number one target of every attacker. I don't care if it's the teenager in the hoodie or an advanced persistent threat. What they want is a valid set of credentials because that gets your foot in the door, your toehold into the door. Every company faces this problem, that way too much of their data is wide open. So any valid credential can get access to it. Once you're in the door and any attacker worth their salt will tell you, if you gave me one valid credential they can Kerberos a service account. They can find the ones that are admins. They can use a Pass-the-Hash, a golden ticket. They can escalate privileges, move laterally. Once they're in the door, it is not hard at all.
The biggest problem companies face aren't passwords on Post-It notes. It's their data. Nobody's breaking into a bank to steal the pens. Nobody's breaking into your network to steal anything but data, their data is wide open. And a whole lot of it is totally unmonitored too, which I think we're gonna get to a little while. That's the biggest problem, and password reuse. It's not the device, it's not the Internet of Things. It's that not enough people use a password manager, not enough organizations enforce multi factor authentication. These are basic security stuff. I mean, I've got friends that are pushing 70 [years old] that were working on Unix in the ‘70s and they'll say, yeah, the biggest problem we have is everybody's using the same password.
We did a risk assessment with a company a couple of weeks ago. And one of the things that we look at is the hygiene and security of active directory.I swear, I can't make this up. We found one of their user accounts, it was somebody who wasn't an executive, but it was an old beard who'd been there forever. And he was there when their active directory tree was spun up, and that account wasn't in the OU, that was getting password policy set. That's a fancy way of saying, his account, he could have any password that he wants, and IT had no idea what it is, and he was never forced to reset it. The password was five characters. Do you know how fast... I don't care what the password is, how fast it takes to crack a password of five characters? It's fractions of a second. Which means, he has had a password that has never been reset that can be cracked in fractions of a second. You can't secure your network when you got credentials that are protected that way.
The real problem is too much data open to too many people. But a corollary that, or related to that, is that everybody is drowning in logs, but they don't necessarily know when a single log item and event is really, really important. Here's what I mean, let's go back to this user that has a crappy password. Let's say he used that same password for his VPN account, or there was a single sign-on so it used the same login and password. This is a huge problem now because every VPN account, every 365 mailbox is constantly under brute force. It's just a fact of life. So his account gets brute forced. Now, a brute force attack is a bunch of failed authentications and then a successful authentication, right? For those that don't know, a brute force is, I'm just gonna try every password, or I might do what's called a password spray. Maybe I found his account in the dark web somewhere as part of a breach. And I've got a password that might work. Anyway, we see a series of failed authentications, and then we see a successful authentication. How do you know whether that's successful authentication isn't just him logging in while he's on vacation? But it's actually a kid in a hoodie or an advanced persistent threat. The answer is, you need to tie that successful authentication to everything else that that user is doing. Let's say Bob here with a crappy password just had his account brute forced. If that account then goes and starts accessing his email and looking at the files that he normally looks at, it's probably not a brute force attack. It's probably just him either mistyping his password a couple of times, or logging in from a new location. But let’s say Bob's account logs in successfully from a place we've never seen before, maybe at a strange time of day. And it's a device we've never seen Bob use, that's odd. Now, Bob is also looking at data that Bob has never looked at before and is trying to authenticate other devices on the network. And Bob's account is basically executing commands like reverse DNS lookups to do recon that Bob would never do. That's how you know that there's a successful brute force. That's how you know that that one event is the most important thing that you need to look at because it's leading to all of these other behaviors that are strange.
MH: Which brings us back to the point that you made about an audit trail and in the acronym, AHAT: Always have an audit trail. But the problem is those log files are massive. And I can imagine IT administrators are getting red lights and alarm bells going off on their screens, constantly. What roles do machine learning and artificial intelligence play in thwarting attacks? And even just making you aware that they're happening,
BV: It's critical. Because as you say, the amount of logging is massive. The ability to combine different kinds of logging information and other metadata is critical. And then the only way to figure out when, to go back to our friend Bob here, that successful authentication is a successful brute force, or is IT just Bob logging in while on vacation? The only way to figure that out is through machine learning. There is no functional or realistic way that anybody's going to ever look at all these logs. Even if you've got technologies like the really powerful SIM, that security information and event monitoring that can actually parse and absorb all of these logs. Unless you're using machine learning to correlate them together in useful ways, you're gonna be drowning in alerts and the other metadata that's really important.
So let’s say you've got an event in a giant log that is Bob accessing a file. Is this something that's normal for him or not? Well, you need to know a lot about this event, if they even have the information at all. A bigger problem is that a lot of actual data access isn't logged at all. But let's say you've got the log. Well, is this an event on something sensitive? Who is Bob? What's his department? What's this location? What's his role? If that event doesn't just have information about the sensitivity of the data being accessed or the role of Bob… Is Bob an administrator? Is that account a service account? Maybe instead of Bob, maybe it's the backup service that you use to access data. What device is being used to access that event? Well, in order to do that, you need to take the IP address and do a lookup of the device name because the internal IPs are going to change all the time. Where in the world is Bob coming from. This is a technique that's called enrichment, and this is related to machine learning, but it's not machine learning. What this is taking different kinds of information like, is the data sensitive? What device is being used? Where in the world is the user coming from? What's the users role? Adding that to the log item so that you have enough information that the machine learning can use it for profiling.
Machine learning is both an exciting and a really scary word for people in technology. Scary because it uses the buzzword as a magic word to solve every problem. But it's really exciting, because it's the only way to solve a lot of these problems if that makes sense. What machine learning depends on is a lot of data, and a lot of clean data. So if you have a lot of log data, that's useful. But what's really useful is the ability for the machine learning to pick out what are called features or pieces of this data that it can use for analysis. So it's one thing to say, I've got a log of everything that Bob is doing. It's another thing to say entirely that I've got a log of everything that Bob is doing that's been enriched with things like classification and location and device names, because then I know what's normal for Bob in really interesting ways. Does he normally touch sensitive data? What device does he normally use? Where does he normally come from? And what does he in his role normally do? Now I can use machine learning to only alert when something real goes on.
MH: What role does the telecommunications service provider play in that scenario? If at all? I mentioned these key areas like consumer devices, IoT, the network itself, or the backend systems. And I asked you, what's the least protected? What's the most attractive? And basically your answer was well, frankly, it's everything.
BV: Yeah, I wish I had a better answer for it.
MH: Where does a CSP start?
BV: Well, the CSP needs to start at their perimeter. If all of my applications are say... And I'm going to pick on Amazon, they run the bulk of cloud computing in the world. So I've got everything in Amazon. I've got services, I've got data, I've got all kinds of things that are running virtualized in Amazon's cloud. As a provider, what is Amazon's responsibility for my assistance? Well, they're responsible for their own perimeter, right? They also should be able to tell me whether there is unusual access to any of my data or systems. But once something is authenticated, Amazon does not have the context about what's going on inside the systems that Amazon is providing. So Amazon has no responsibility and they will tell you as part of their licensing agreement, that there's nothing that they can do if I leave my systems misconfigured or unlocked.
The answer to your question is the CSPs absolutely have some responsibility because they are providing the infrastructure, but it can't be their total responsibility. And in fact, it's not even their primary responsibility to make sure that your systems are secure or that you can detect abnormal behavior or some sort of attack.
MH: So you don't think that the CSP has a role to play when it comes to application security? Like it's hosting the client app, but it's not their app.
BV: They do have a role to play because when you're putting something in the cloud, you're putting it on somebody else's internet connected computer. Which means now they are responsible for the perimeter. The perimeter is one important piece of security, but once you get past the perimeter, and this was true before cloud computing, when we were all running in data centers. We had this notion of, I used to call it the candy bar defense, where if I have a big and powerful firewall to keep all the bad guys out then I'm good. But the problem is that once something bypasses the perimeter, the hard outer shell of your candy bar, it's chewy, soft nougat inside. Somebody gets past the perimeter, it's open season once you're inside.
The CSPs are providing the perimeter and they need to keep that perimeter hard, but it is the responsibility of whoever’s owning the application to make sure that it is not a soft chewy center inside. That they are doing logging and they have proper authentication, authorization, detective and preventive controls to make sure that when something is past that perimeter, that they can detect if something goes wrong.
One of the interesting things about 5G is that it dramatically complicates or makes more complex the infrastructure.
MH: Who becomes the greater target as 5G becomes the norm? As it replaces 4G? who's the greater target, the provider of the services or the companies leveraging them?
BV: A good question. And I'm forced to say six one way, half a dozen the other. I'm not sure there is a greater target. Infrastructure is always going to be a target. And one of the interesting things about 5G is that it dramatically complicates or makes more complex the infrastructure. What 5G does is it makes it really easy or possible really for orders of magnitude more devices to be connected and to be part of your infrastructure and part of your network. So instead of having one or two or a small number of gateways to a manufacturing environment, for instance, suddenly every single piece of that manufacturing environment can be a gateway and can be connected. 5G opens up a lot of targets. It's kind of like the Internet of Things of things almost, because there's so many more potential entrance points into a network. I don't think there is a single answer that it is the provider or the application owner. I think it's both.
MH: If 5G is replacing the last mile of twisted copper or cable using Wifi, are consumers and consumer grade hardware, the weakest link?
BV: They have been for a while and they probably will continue to be. We saw a rash of issues when home wifi routers weren't shipping with randomized passwords. And even now, home wifi network routers are one of the easiest entrance points into a corporate network.
One of the stories that I like to tell is there's a CSO that we work with, who said, "At the beginning of the year, I had five office locations and about 1500 users, which is a midsize enterprise. And suddenly in March, I went from five locations at 1500 users to 1500 locations each with a single user, every one of those locations are those users' home wifi networks, which of course we had remote workers before, but it wasn't 100 percent of the time. So suddenly I have to worry about 1500 different home wifi networks that we have no effective control over at all. It’s not just the corporate laptop on these networks that my user is using. It's their spouse and their kids. It's all of their Internet of Things on their home wifi networks. Suddenly that's part of the attack surface area. And it's not just a cure there, people are working remotely or certain times a day. This is the new constant now. So now I am suddenly trying to secure a much broader and more complex attack surface area.”
I don't think 5G is gonna make a huge difference for a lot of consumers right away. I think the bigger deal with 5G isn't necessarily consumer grade hardware right away, although it will be eventually when every single thing in your house has got a Wifi chip in it. But what it's going to do is make it easier for every single thing in everybody's house to have a wifi chip in it. And we have to remember now, after March, everybody's house is now part of our corporate security environment. This is why audit records and machine learning for behavior are gonna be more important than they've ever been before.
MH: How has COVID-19 changed security?
BV: Two things, One is the scenario that I just described with suddenly everybody's home wifi network is part of our corporate security environment. The second thing is, and this goes back to what we've been talking about previously. There's been a rapid acceleration of data workflows systems into the cloud. The biggest kind of fundamental way of thinking about the change after March or after COVID is that now there is an expectation that anybody, any one of your users should be able to get access to anything that they need to do their job from any device, wherever they are in the world at any time. What that does is this notion of constant cloud-based collaboration. I need to be able to get access to any data from anywhere from any of my devices and I need to be able to share it with anybody else if I need to. COVID hasn't just opened up the attack surface area. It's made the attack surface area bigger because users are now empowered when you're using technologies like Office 365 to collaborate, to grant access to people inside and outside the organization, in order for them to do their jobs.
In a past life, I worked on a help desk. And the way that we would secure data in the data center is we'd create a shared folder and we create an active director group and put people in that group. Sit that group on the folder. And now there's a lot of problems with that. I mean, you can have data that's open to everybody cause of a misconfiguration. You can have multiple groups that grant access to that data, but the users themselves can't really break your security model. Office 365 and cloud collaboration in general flips that on its head because users are doing the sharing. They're sharing access to not just folders, but entire SharePoint site collections and team sites and individual files. They're sharing it with people inside and outside the organization or creating links that anybody can see. And they can do this from web browsers, and mobile apps and desktop clients and teams clients from anywhere at any time. Suddenly from a security perspective, your users get to decide who's got access to what whenever they want. It's functionally impossible without the right kind of controls in place, additional controls to make sure that only the right people have access to what they're supposed to. And that goes back to the biggest problem that every company faces, too much data open to too many people.
In a post-COVID environment, everybody can open up anything they want with anybody. We're seeing a lot of organizations struggle with this. One of our customers, in the beginning of the year, in January, they had a three year plan to move their on premises, data and collaboration workflows into Office 365 using Microsoft teams. And by the way, at Varonis we use Teams and 365 extensively so I'm not here to knock it. So they had a three year plan to move it to migrate into office 365. And in March that three year plan turned into a three week plan. When you do this, regardless of the pace, there is a whole host of security and configuration challenges that you're going to have. And what COVID has done is rapidly accelerate that change. We've been talking about digital transformation for 10 years. It's happened now. There's gonna be no such thing as paper collaboration anymore. Everything is gonna be cloud-based. Everything is going to be mobile-based. And that transformation, or that transition to that model to get back to what you really ask what's the biggest change that transition has happened faster than most companies were prepared for. Many companies, of course, ours included, we're a tech company. We were fully remote and fully immobile, and we have been since the company was formed. But not everybody's like that. And what this has done is, especially for a lot of industries that weren't prepared for this, what it's done is it's rapidly accelerated that transition. So it's forcing everybody to think about exactly the kind of issues that we're talking about.
MH: Tell me about the toehold in the door. I know my front door lock can be picked using a kit purchased off Amazon.com. But I still sleep well at night because I know that lock is meant to thwart crimes of convenience. People jiggling the door to see if it's unlocked. If someone really wanted into my house, that lock’s not going to stop them. Are CSPs sleeping well at night? Is there a digital version of a beware of dog sign that's giving them a false sense of security. What kind of security measures actually make hacking less attractive?
BV: It's gonna sound boring. I wish I had a really super technical answer for you that would wow everybody and say, "Oh, that's a really good idea." The answer is honestly the most common attack that we see when you're talking about remote attacks from cyber criminals and APTs into enterprise environments is brute force. Honestly, it's going and finding a password dictionary and brute forcing your way into a network.
One of the variants of malware that we discovered last year to move internally on a network, and it got in by a phishing, of course. Once it was inside and established a foothold on the device, it used a login and password dictionary to move laterally around the network. What the malware would do is automatically try to authenticate to all of the devices that it could see locally on the network nearby. And then once it was on another device, it would move to another device and move to another device. And the malware itself was called Cuba. And what it would do is harvest financial credentials and put hooks into web browsers. It was a really interesting variant of malware. But that aside, the reason I'm telling you this story is that the login and password dictionary that was used to successfully move laterally around the network of upwards of 3000 different enterprises. We know this because we were able to access the proxy servers... The proxy of the command and control server that this malware was phoning home to was 12 logins and 300 passwords. That's it. You do not need a dictionary of 45 million terms. Now this is an internal authentication, which means that my answer to your question of multifactor authentication doesn't work as well. That's the ‘Beware of Dog’ sign. You make sure that any remote connection to your network or to your cloud environment requires multi factor authentication, and thankfully, we're moving to a world where people are getting used to it. I log into my banking website and then they send me a text on my phone. We're getting used to biometric authentication, like fingerprint readers and face readers, being part of our authentication, multi factor authentication that is the ‘Beware of Dog’ sign. You said it yourself. You lock your door. You sleep pretty well at night because burglary is a crime of convenience. You don't have to have the best bike lock. I'm a cyclist. I'm always locking my bike up here in New York city. You don't have to have the best bike lock. You gotta have a better bike lock than the guy next to you, right? So multifactor is just making it a little bit more difficult. Now, multi-factor is not going to stop a determined adversary. As you said, somebody really wants to break into your house, they're gonna go buy a lock pick kit. They're going to get in if they want to, there are lots of ways to bypass multi factor. But that's the ‘Beware of Dog’ sign.
MH: So it sounds like in any security system, the weakest link continues to be people.
BV: Yeah, that's such an easy platitude. It really is. It sounds interesting and scary. The weakest link is always people partly because you can always trace back any misconfiguration or any problem to people. Social engineering really, really works. Phishing works. There is no organization that is phishing proof. I mean, maybe there's some defense contractors that are but anybody who's ever been in IT or security and has run a phishing simulation internally knows somebody is gonna get caught. I think on average, it used to be 25 percent. I think that's down to about 10 or 15 percent now. But if you run a phishing simulation, which is good, right? That's a big improvement. But if 10 percent of your users click on a phishing email, that's a massive hole.
MH: So you're saying is you can't firewall stupid.
BV: That's okay. I wanna empathize with everybody out there for a second.
MH: Okay. you would've been phished yourself. Haven't you?
BV: We did a phishing test internally once before COVID I traveled four weeks a month. So I, my expenses are always just a, it's a huge exercise for me to actually do my expenses. And I got an email one day, I was standing in my backyard and I got an email and it said the expense was rejected. And I was like, what? I clicked on it and I immediately said, you have been hit. It was an internal phishing simulation. I do this for a living and anybody can get hit by this stuff. It's not necessarily stupid, and what we're seeing is the determined attackers, spear phishing is a real thing. Like it is not hard to go on LinkedIn, figure out the people that somebody works with craft an email that looks totally legitimate and then spear phish them. And that does not mean that that person is stupid. It just means it's not that hard to get past a human brain.
MH: Brian, fascinating. Thank you so much for your time. I have to go now and lock down my wifi.
BV: If you haven't done that, you absolutely should.