Tag: Governance

What Happened at Capital One?

What Happened at Capital One?

by and | Organizations, Technology

There have been many words written about the Capital One breach – but a lot of them didn’t explain what actually happened. We care a lot about security in general, and cloud security in specific, so Josh set out to find some words that did explain what happened:

The Krebs article might be the best for this. However, as far as we could tell, no one’s tackled it from a “what can enterprises learn from this?” standpoint, and…that’s what we really care about.

TL;DR: The Event

A hacker named Erratic, who was a former AWS employee, took the following actions:

  1. Owned the Web Application Firewall (WAF) on Capital One’s Amazon Web Services (AWS) Virtual Private Cluster (VPC)
  2. Used the credentials of the WAF to connect to other AWS resources in the VPC, including their storage objects (S3 Object Stores)
  3. Synced (copied) the object store to her own storage

“With this one trick, you can get 100M Credit Card Numbers! The secret THEY don’t want you to know!”
– Best ClickBait Ad Ever

…ELI5: The Event

So…there’s a lot about the mechanics of this that’s unclear. But we can explain what seems to be widely accepted as fact. First, some definitions:

  • A Web Application Firewall (WAF) is basically an entry point into a system – it isn’t intended to be entry, though, it’s intended to be a layer of defense.
  • AWS is Amazon’s public cloud.
  • A virtual private cluster (VPC) is a cordoned-off part of a cloud – so, it was an area of AWS that was specifically for Capital One.

So…

  1. Somehow the hacker Erratic was able to log in to one of Capital One’s WAF.
  2. From there, she got to their storage objects that represented information about people – specifically, people who had used the business credit card application…application. Overloaded words are the best!
  3. Finally, she copied those storage objects that represented people to her own area of AWS – like copying a file from someone else’s Google Drive into your Google Drive.

Questions Outstanding

…there are a lot.

It’s not clear to how Erratic did #1, logging in to the WAF. The most likely answer is that the username/password was something not complicated enough – like maybe the default of admin/admin. But there are also other possibilities, and if Capital One has disclosed this piece, we couldn’t find it.

There are a few ways step #2 could have happened – the WAF could have already had access to all of the storage objects, or Erratic could have given the WAF direct access to the storage objects. The J Cole Morrison article above explained one possible scenario: Amazon IAM could have been used to take advantage of the fact that she was already in the WAF and then extended the default trust of “well, you’re in the WAF, so okay” – security people call this a “pivot”.

Step #3 is basically…copy/paste. There are probably some interesting nuances here, like…if she didn’t give the WAF authority to read the objects, why did the WAF have the authority? What business use case would require giving an access point read access to an entire store of customer data? Also she would have had to have given something access to write to her own AWS space, at least temporarily.

The Pain: $100M-$150M

The Capital One press release stated that this incident will “generate incremental costs of approximately $100 to $150 million in 2019.” Capital One was one of the earliest companies to go to AWS/the cloud, and they made a lot of noise about it – here, and here. Explaining technology success is one of our favorite things, but there are trade offs if you could otherwise manage to keep your backing infrastructure a secret.

This has lead to egg on AWS’s and Capital One’s faces, which is unfortunate, because this really doesn’t have much to specifically do with AWS or clouds in general….

…or does it?
– Not Intended to be ClickBait

Clouds in General

This isn’t the first AWS data breach (see end of the blog for a list of others). The list is not small, unfortunately.

Please raise your hand if you are sure you haven’t been hacked?

We’re gonna say this is partially because AWS is the biggest, been around the longest, and had to figure out hyperscale stuffs without anyone to copy from because they were the first.

But still… yikes.

A big part of this is that Amazon makes things super easy. So easy a caveman could do it, right? And…that’s the trick. It’s super easy to type in a credit card (or even an AWS gift card, I (Josh) have one they gave out at a trade show) and spin up some storage and compute. Unfortunately, it isn’t super easy to spin up security tailored to clouds.

We used to have to wait for infrastructure teams in our data centers to (hopefully) handle the security for us. They’d put your request in a queue, and get to it a week later…then they’d ask the storage admins and VM admins for some data and some compute, and that request would go into a queue…and then, several steps later, the firewall admins would get involved…but doggone it, eventually the pros would secure things like they were trained.

VM-based infrastructure has been around a long time, and the kinks have been worked out. Cloud infrastructure is newer, and exponentially faster to use – that’s one of the biggest appeals. Unfortunately, because it’s newer and because it’s so fast, kinks still exist – and one of the biggest is how to make it secure without slowing down the people using it.

Clouds are not all insecure, the sky is not falling – but they do require more deliberate attention to security than perhaps we’re used to in most of IT.

Takeaways and Recommendations

With Infrastructure as a Service that’s as fast and easy as cloud-based, it’s clear that there are often times when the right security-aware folks are not involved. It’s extremely easy to get going with platforms like these, which is…kind of the point. Simply put, you can get insecure systems faster and easier than you can get secure systems – for now, anyway. The industry knows this, and is trying to make it better.

Until security catches up to the speed of IaaS, companies need people who can secure their systems to be involved in setting up new platforms, and setting up best practices for their use. The balance point of that is not removing too much of the speed and agility gains of advances like IaaS because of security – ideally security should be something that everyone agrees is worth the trade.

So…after all of that, here are some recommendations:

  1. Single layers of security are not enough. You need Defense in Depth, and vital areas like customer data need to be strongly protected regardless of the platform trying to access them.
  2. Security practices and implementations should be transparent, at least within a company, and questions should be welcomed and encouraged. Open culture helps with security, too.
  3. Security should be automated as much as possible, and that automation should also be transparent (infrastructure as code).
  4. Enterprises need to choose platforms that are secure, that have people dedicated to the security of that platform as their job.

Other AWS Data Breaches

Lean Enterprise Innovation

Lean Enterprise Innovation

by and | Leadership, Organizations, Technology

(FYI, we’re using affiliate links to Amazon in this post!)

Technology innovation is vital. It can enable business success – and it can also drive business innovation.

If a business falls behind the technology curve, it opens itself up to the risk of under-serving its customers, and eventually being out-maneuvered and defeated in the marketplace. This happens over and over to businesses, where a competitor’s technological innovation pushes them right out of existence – see Blockbuster (Netflix), and Border’s (Amazon, B&N). Amazon has also innovated while JC Penney has stagnated – department stores could have taken the world by storm via the internet – but their online presences weren’t good, certainly not as good as their competitors, and therefore neither were their sales figures.

…

Read More Read More

Libertarian Enterprise Governance

Libertarian Enterprise Governance

by and | Leadership, Organizations, People, Technology

Be Good

There is freedom and peace and pride in being truly good at something.

Not to seem good.

Not to check off the boxes of good.

But to actually be good.

Everyone should have the freedom to be actually good.

Establishing governance philosophy in an enterprise is a tricky combination of “how much do we trust our people?” and “okay, but how much do we actually trust our people?” The fact is, people are going to screw up. Some of them might deliberately try to screw something up, but mostly they’ll just make mistakes. If the organization can’t handle mistakes, that’s a problem with the adaptability and flexibility of the organization, not with its people. This means that people’s potential mistakes are not a reason to default to not trusting them.

…

Read More Read More

Building Reliability

Building Reliability

by and | Organizations, People

The definition of reliability for the purposes of this post: other people believing that you (or your team) will:

  1. do what you say you’ll do, and
  2. do what they expect you to do.

Reliability is other people believing that you will do what you say you will do and also what they expect you to do.

Why Reliability?

Reliability is vital to several areas of organizational success:

  1. Successful delegation
  2. Independent execution by teams
  3. Independent execution by individuals

Delegation

When you hand off a task, it’s ideal if you no longer need to think about it – and the more you reliable the person you’re handing the task to, the more likely this is. With reliable people and teams, the default assumption is that any delegated task will be completed. Wouldn’t that be amazing? Delegation backed by a perception of reliability actually clears brains of stress and cognitive dissonance. Basically, people can stop worrying and focus on the work they need to do.

This magical theory applies to teams as well – if a team does the work they own, and they believe other teams will do the work they own, then all teams can focus on their own work. Ultimately this eliminates time spent on “following up” (pestering), and lowers everyone’s stress.

Execution

Individuals and teams who are allowed to execute without being required to explain every step and justify every decision can focus more on doing actual things. Even in the awesome Utopian Land of Reliability, some checking in is necessary, for a lot of reasons – one of the best is to give the leader the opportunity to teach and to reinforce good judgement. However, guidance shouldn’t be a nuisance – it shouldn’t turn into oversight to the point of preventing productivity.

If brains are calm and hearts aren’t worried, people can more easily focus on their work, objectives, and goals – and they execute their tasks more cleanly. When each person or team believes that they can focus on their responsibilities rather than worrying about other people’s execution, this leads to overall organizational success.

Reliability seems awesome! Uh…now what?

Cool, you want to be thought of as reliable so you can do stuff. A key question now becomes: how do you build a reputation for reliability? Or maybe the question should be, can this be deliberately built? Nope! Blog post ends here. Thanks for reading!

Kidding.

Building a reputation for reliability is surprisingly simple. In any relationship, you start at the ground floor, with basically zero reliability. To increase the number of reliability points, it’s like a ramp – a slow and steady upward movement.

The super secrety secret to building a reputation for reliability: make a commitment. Fulfill the commitment. Repeat.

Reliability for Newbs

Start a meeting by saying, “I’ll make a recap.” End the meeting, re-iterating that you’ll make the recap. Then make the recap. Tada! You’ve done a useful thing that you promised to do. Congratulations, you now seem more reliable! People now know that you kept that commitment.

You can do this by doing anything useful – say you’ll do it, and then follow through. For bonus trust points, promise to do it by a certain date and then deliver on or before your deadline.

Reliability Level Up

To continue to build your new reputation for being reliable, continue to keep your commitments. Small commitments can build reliability only to a certain level –  but after keeping enough smaller commitments, people will begin to give you opportunities to keep larger commitments.

These larger commitments, once kept, will give you more reliability points – but they’ll also be more complicated, and take longer to complete. However, in order to build to a high level of reliability, the pattern is exactly the same: steadily fulfill commitments over time.

Team Reliability

Teams can do the same thing, for themselves and for the teams and people they work with. At any time, an individual on the team can step up and start keeping commitments on behalf of the team. This results in a team’s reliability points increasing. The individual can then teach the other team members to keep commitments on behalf of the team. Every commitment kept adds to the team’s reliability point total – and also helps boost overall team morale.

There’s also the team’s perception of the team’s reliability – when this is working correctly, it’s usually when people say things like, “it’s a good team to work on.” Goals are a great way to help build this – committing to goals has the added bonus of building unity, and then following through on those commitments builds reliability. Even if a goal is not achieved, the tasks necessary to attempt to fulfill the goal are completed, and those commitments are kept.

Uh…Whoops. I Broke It.

A reputation for being reliable can be broken. To break it, don’t keep your commitments. Promise things that you can’t or won’t do. You could also just stop doing the things already expected of you – because reliability is both the things you say you’ll do and the things people expect you to do.

Let’s say you fail to fulfill a commitment and you lose reliability points. It happens. People aren’t perfect – but that’s okay, because perfection isn’t necessary here. There are two things that can help repair this break and then allow you to go back to building reliability:

  1. Accept responsibility. Clearly. Be obvious that you accept responsibility for the failure. Hiding from a commitment, hoping no one notices it wasn’t fulfilled, does even more damage.
  2. Renegotiate the commitment. Make a reasonable judgment about re-committing to the task. Does the task still make sense to do? If so, what is a reasonable deadline for the commitment? If the commitment should still be fulfilled, fulfill it.

Ultimate Value

Being perceived as reliable lets people and teams believe that they can do increasingly harder and more awesome things. If we don’t truly believe that we can succeed, we don’t try. When we do believe that we can succeed, we start to watch for opportunities to do increasingly more awesome things.

Reliability lets teams believe in themselves.

Reliability also lets us focus on execution instead of worrying. We can focus more on tailored challenges and working hard, and less on trying to split our focus to all details of all things.

Basically…when we believe that we can succeed (more awesome things), and we can focus on execution (get things done), more awesome things get done. Reliability helps fulfills both of those requirements.

Reliability makes more awesome things get done.

Meta