Review of SANS SEC760 - Advanced Exploit Development for Penetration Testers

A little over a week ago I wrapped up taking SANS Advanced Exploit Development for Penetration Testers (SEC760) at SANSFire 2015 in Baltimore, MD. This class is a 6 day long class that covers advanced Linux exploitation, patch-diffing and one day exploits, Windows kernel exploitation, advanced Windows client exploitation, and a variety of other topics. The end of the class, in typical SANS penetration testing class fashion, culminates in a capture the flag (CTF). So far I think this has been the best SANS class that I've taken. Prior to SEC760, I took SEC560, and SEC660 (course review here). The material is extremely technical, relevant, educational, and interesting. The course's author and instructor, Stephen Sims, has really put a lot of time and effort into making this class very challenging yet approachable.

Day 1 - Threat Modeling, Reversing, and Debugging with IDA

Day 1 of the class starts you off with a dive into the world of threat modeling. I'm sure that many people who end up taking SEC760 looking to learn how to advance their exploit writing game aren't exactly interested in talking about threat modeling. I was one of those individuals, but I completely see the value of threat modeling from the stand point that you as an individual can provide an extremely valuable resource to your company when correctly implementing processes like the Microsoft Security Development Lifestyle (SDL). Not to mention, an individual that can provide effective threat modeling stands to make decent money with that skill.

Following the threat modeling, the technical aspects of the class begin to pick up. The rest of the first day is spent on going over the process of reverse engineering binaries and becoming familiar with IDA. For those who haven't used IDA, this day provides you with the opportunity to become familiar with the interface and capabilities of the tool. I should caution, however, that if you've never used IDA, you will not be able to take the first day to learn it. IDA is an extremely complex tool that requires a lot of time to learn. Having experience with other debuggers such as OllyDBG, Immunity, or x64dbg certainly helps (and is expected for this class), but there is no replacement for actual experience.

Day 2 - Advanced Linux Exploitation

Moving on to Day 2 of the course, the class explores advanced exploitation concepts for Linux. The first topic of the day is focused on getting the students more use to heap based exploitation techniques by learning about the original exploitable flaw in the dl-malloc unlink() macro implementation. From here, the day progresses to learning about exploitation examples in custom memory management implementation, function pointer overwrites, and format string vulnerabilities.

I personally would have liked to have seen more Linux exploitation topics, but given the focus on modern exploit development, I understand why Stephen decided to focus more on Windows exploitation for days 3 - 5. To be fair, a lot of the vulnerability and exploit categories taught on days 3 and 5 can be applied to Linux, albeit with some modifications. This opinion is nothing more than a personal bias.

Day 3 - Patch Diffing, One-Day Exploits, and Return Oriented Shellcode

Day 3 starts off with a refresher into ROP based exploit development (a pre-requisite for the class). In these refreshers, you use ROP to disable exploit mitigations. With return oriented shellcode, you learn how to make all of your shellcode execute via a series of ROP gadgets. This has the added benefit of being able to bypass certain exploit mitigations even in instances where common bypass techniques are not available. 

From Day 3 on, Stephen puts a lot of focus on the art of patch diffing. Patch diffing is the process of looking at an original un-patched binary, and comparing the old version with a newly patched version. By using these diffing techniques, it makes it easier for a researcher to pin-point the area within the binary that changed. This is extremely helpful in developing 1-day exploits. For this part of the class, I used a new tool, Diaphora, to perform my diff-analysis. Diaphora worked out great for me. Diaphora is able to perform patch diffs at the control flow graph, assembly, and pseudo-source code levels (if you have the Hex-Rays decompiler plugin). And the best part, Diaphora is free, open source, and actively maintained.

Day 4 - Windows Kernel Debugging and Exploitation

Moving along the Windows exploitation ecosystem, Day 4 focuses purely on Windows kernel debugging and exploitation. I really enjoyed this section of the class. A recommendation I would make for those who are considering taking this class: if you take this class, make your life a little easier and install Windows as your host OS and use VirtualKD by SysProgs to communicate with your target Windows virtual machine. This class has a lot of moving parts and the more you can simplify those pieces, the more time you can spend actually working on the exercises and by consequence, the more you learn.

This is the day when students will become much more familiar with Microsoft's WinDBG. Prior to taking this class, I had never touched WinDBG. Although I found the learning curve rather steep, I came to appreciate the power provided by the debugger. Day 4 culminates in an exercise where you will learn how modern privilege escalation works on Windows via a technique known as token stealing.

Day 5 - Windows Heap Overflows and Client-Side Exploitation

Day 5 of the class zeros in on the area of advanced Windows client side exploitation. After getting a deep dive into the internals of Windows heap management, the entire day is spent learning how to find, identify, confirm, and eventually, exploit modern use-after-free bugs. After going through this day you will be equipped with the knowledge of how to perform modern exploit development on Windows 7 and 8 on applications such as Internet Explorer. Additionally, Stephen goes over the next steps to take when developing exploits for Windows 8.1, exploits that bypass Microsoft's EMET, and exploit tricks that still work with the latest preview version of Windows 10.

I will caution the reader, however, that during my time taking the class, we didn't actually perform any exercises on exploit development for Windows 8.1 or above, and for good reason. Even with the normal class hours of 9am-5pm and the extended boot-camp hours, there simply isn't enough time to go from just learning what a use-after-free exploit is to developing exploits that work on all versions of Windows and making those exploits bypass any new exploit mitigations that may be in place. Overall, Day 5 is where you really begin to have all the pieces from earlier in the week align. I found this day to be the most interesting and I'll definitely be returning to the material in order to ensure I've mastered the techniques presented.

Day 6 - Capture the Flag

For the final day of the class, the class is split into teams to compete against each other in a Jeopardy style CTF. There are 16 challenges with a total of 4000 points up for grabs. I won't go into detail of the challenges themselves, but the competition requires your team to exercise all of the skills gained through the previous 5 days. In the end, our team successfully completed 14 out of the 16 challenges to earn a score of 3200. Although the game was very competitive, 3200 points was enough for our team to come out on top. As a result, we earned the coolest coin offered in the SANS series of challenge coins. The 760 skull coin.

Team ROPNOP Wins the Skull Coin

Conclusion

Overall, I loved this class, It was an absolutely great experience. I encourage any individual looking to take their exploit development skills to the next level to take this class. I will warn you, however, that this is not an easy class, You can expect to be up late into the night after each day of class if you want to finish all of the exercises. Go into this class expecting to work hard, get frustrated, and exercise your patience. This is not a class for beginners. If the term Return Oriented Programming (ROP) is something you aren't familiar with, GDB is a foreign and feared tool, the heap sounds scary, and kernel debugging is shrouded in a fog of wizardry, you may want to consider taking SANS SEC660 first. Otherwise, embrace the challenge and I'm sure you'll finding it very rewarding!

Palmetto Cyber Defense Competition - 2015

Last month my fellow employees and I wrapped up our third annual Palmetto Cyber Defense Competition (PCDC). Inspired by the Collegiate Cyber Defense Competitions, PCDC has been a computer security competition for high schools and colleges in South Carolina. I've written about the event itself in the past PCDC-2014 so I won't go into detail about what the event actually is. Those details can be found at the official PCDC site. Instead I wanted to focus on a couple of the things that make PCDC unique, lessons learned from putting on a computer security competition, and where we are going in the future. For those of you looking for a detailed Red Team write up, that can be found here. I took the time after the competition to create an in-depth analysis of the Red Team's process for those looking to learn from their mistakes at this year's competition. The rest of this blog post is a few of my thoughts from an organizer's perspective and not from the perspective of the Red Team lead.

First off, PCDC is unique in that for the first two years, both high schools and colleges competed. Each group had their own dedicated competition day which meant that the PCDC competition was actually run twice. This means that after the first day, all the machines must get reset, re-imaged, and configured slightly different for the next day. This year, we had a third day added; a professional day. For the first time, PCDC was to be run three times. The schedule of events was as follows: first day was high school, second day was college, and the third day was for professionals. For the professional day, we had groups representing a mixture of government and private industry. From the government side teams were comprised of members from the 24th Air Force and U.S. Cyber Command while the industry teams had members from Scientific Research Corporation (SRC), and SPARC. All in all, we had 4 government teams and 4 teams from industry.

Our goal from the beginning was to design the competition network to be believable and realistic. Since it was not known to us during the planning and design phase of the competition that there would be a professional day, this year's theme was based around an online game development company. Each of the Blue Teams would be responsible for making sure their development company continued to function through the course of the day and deliver the latest version of their game to their user base through audience accessible tablets connected to each Blue Team via a dedicated wireless connection. One of the things that we quickly realized was that our ambitions greatly eclipsed the amount of time we had available to create the network. Remember, a decent portion of our time goes into infrastructure development so that the competition can be rerun the next day. To add to that pressure, we do not have control over the facility in which the competition is hosted. As a result, or preparation time from the end of one day the beginning of the next is usually around 3 hours.

To put it simply, there was a lot of different attributes that we wanted to include into this year's competition, but we ran out of time. One of the biggest things we feel these types of competition lack when trying to simulate real world networks is realistic user activity. This year we attempted to remedy that by developing simulated users. We got all the code developed and tested for the user simulators, but due to a hardware failure, we were unable to deploy them to the competition network this year. In the interest of education and sharing, I have opened sourced the code for the user simulators on GitHub. We'd really like to hear back from anyone that is doing something similar.

A few of us have been involved in multiple CCDCs and PCDCs and every year, we make the comment that the scoring system needs to be altered. Although this was in the works before this year's competition even took place, we haven't had time to finalize what fixing the scoring means. At this point, I think we have a much better idea of how we are going to fix the scoring. To highlight why scoring is such an issue, I want to talk about how winning Blue Teams typically approach this competition. Within the first few minutes, the strategy includes removing all unnecessary accounts, changing all the default passwords, and for some, unplugging their entire network while they continue to harden. Now, from a strategic perspective with the goal to win a game in mind, I can't argue with this approach. The issue that I do have, however, is that this leaves the networks in a pretty unrealistic state.

Each Blue Team is given an information packet at the beginning of the competition. In that packet includes the names of the accounts that the automated scoring engine will use to log into their systems and perform service checks to make sure the Blue Teams still have their services up and running. Once these account names have been identified, the Blue Teams will delete every other account off the workstation or server. This means you could have 3 or 4 domain joined Windows workstations with zero user accounts and only the scoring engine account. It is important to note, that the Red Team is not allowed to leverage the scoring engine account to gain access to the Blue Team's networks. It's also not realistic. A computer security competition should force the students and competitors to perform real security tasks with the presence of real users. Now, since this is a competition and getting that many unbiased volunteer users is unrealistic, we need simulated users.

Other areas where improvements to scoring need to be made is in the way the scoring engine actually evaluates successful checks. Up to this point, a common service to check for is a functioning MySQL database. Typically the scoring engine will login to the MySQL database, make a query, and check for a specific key-value pair or for the presence of a specific table. This simply isn't good enough. For a real company, the database needs to have constant transactions generated by realistic activity. Right now, Blue Teams get away with making a backup of the database in the beginning of the day and just restoring it anytime the Red Team deletes the database. As long as the Blue Team restores the database in between scoring engine rounds, the scoring engine gives that Blue Team a perfect service check score. Now, this is slightly cured by the fact that the Red Team reports incidents to the Gold Team and the Gold Team can decide to take away points, but these types of scenarios need to have bigger impact on the 'day to day' operations of the Blue Teams' networks.

Where we plan to go in the future will attempt to combine 3 facets of scoring. The first being financial. The scoring engine will no longer add points, but will score the Blue Teams' companies in a financial sense. The second facet is from an internal employee and systems perspective. Employees must be able to perform their job related duties and interdependent systems must be able to communicate with each other. Finally, the third facet is from the Red Team. This year we tried something new by giving the Red Team specific targets/flags to capture when we gained access to the the Blue Teams' networks. This included the credit card numbers in their customer database, the source code to their latest game, and a few other  things.

Now, I know some people will argue that the CCDCs and even PCDC already take these things into consideration with the scoring engine, but we argue, it is not taken into account enough. The example we like to use is the one where the Blue Team unplugs their network. Now sure, they aren't getting any points from the scoring engine, but in the real world, you can't just go and unplug your entire company from the network. Not only would you be losing sales, but you're paying your employees to do a job that they can't accomplish. And not to mention, the security or IT department has no authority to make that type of decision.

We have thought long and hard about the scoring and we think we have something new and exciting for next year. I don't want to give away too much here until things are more settled. Additionally, we want to find a way to make the audience understand what is going on. PCDC is free and open for the community to come in and view. This year we attempted to show what was taking place by visualizing the the traffic between the Blue Teams and Red Team in real time. I wrote the code for this and am also releasing it on  GitHub. You can see a video demo of it on YouTube.

We have a lot of exciting things planned for next year's PCDC! Stay tuned for more, and if you have any feedback from this year's we'd love to hear it.

South East Regional CCDC 2015 - Red Team

This time last week I wrapped up Red Teaming for the South East regional Collegiate Cyber Defense Competition (SECCDC) for 2015. The SECCDC is special for me for a few reasons. It was my first exposure to the whole CCDC arena, many of my close friends form the Red Team, and the CCDC national champs from last year are from this region.

This year's scenario was similar to last year's. The Blue Teams were responsible for maintaining the operational status of the HAL business network while completing a series of business related injects. The network layout changed a bit from last year, however. This year all the Blue Teams had a few machines that were public facing, and a group of privately networked workstations. The public facing images were comprised of a pfSense software firewall, 2 SuSe Linux boxes, and a Windows 2012 R2 server. The SuSe boxes were used for backup DNS, MySQL database, and the e-commerce web server while the Windows box was primarily performing the normal functions of Domain Controller and primary DNS.

After scanning the networks, we quickly determined that the Blue Teams were running all of their public facing services off of an ESXi server. Additional investigation revealed that the ESXi servers were version 5.5.0 and vulnerable to Heartbleed. This vulnerability became our primary attack vector. By leveraging Heartbleed, we could force the ESXi servers to leak the login credentials in clear-text whenever the Blue Teams logged in. Once gaining root access to the ESXi servers, my goal was to gain access to the domain controller. This is a little tricky when you want to go unnoticed. We were able to jump on a couple of domain controllers that Blue Teams logged into, but left the console open and unattended. The Linux boxes were much easier to compromise. Either they were logged into as root and we could change the password, add users, start SSH, lock the console, and log in remotely, or, we reboot the machine into single user mode, changed the root password, and rebooted. Finally, by leveraging a combination of default credentials and unattended console sessions, we leveraged the pfSense firewalls to lock the Blue Teams out of their networks by turning off the internal interface, but allowed us in via the external interface.

Throughout the course of the competition, the Blue Teams started to slowly kick us out of their networks. This forced us to start getting more creative with our access methods. The first area we looked at was the WordPress site running the e-commerce server. Although we couldn't take advantage of any vulnerable plugins, we could take advantage of the fact that the WordPress configuration let anyone register an account. Now, registering an account in and of itself isn't exciting, but what is exciting, is the fact that WordPress emails you your password. This is important because we had read/write access to the MySQL database backend of WordPress. In the database we could clearly see the administrator's hashed password. Now that we created our own user, we could also see our hashed password in the database. The next step was just receiving the password generated by WordPress. With the Blue Team's network configuration, the WordPress instance couldn't email out to a public email address. But what it could do, was pass the email along to a local Python SMTP server that we controlled. During the user account creation process, we simply specified our email address as 'user@<ipaddress>' rather than providing a public domain name. This worked like a charm. Now that we had a clear-text password and a corresponding WordPress password hash, we leveraged our read/write access to the MySQL database to overwrite the administrator's password hash. Now we could log into the WordPress administrator's account with a password that we knew.

Our other attack vector was actually found by digging through the pfSense source code. For a few of the teams we still had authentication access to the firewalls, but the web administration had been shutoff. It turns out that there is an XML RPC in pfSense. Like I said, we had the username and password, but couldn't turn on the web console and not all the routers had SSH enabled. So we created our own shell. Using the XML RPC and a little PHP voodoo, we pulled down a PHP shell and created our own web console. In most of the Red Team's opinion, this was one of our coolest finds of the event.

One of the largest differences I noticed this year was how a couple of Blue Teams were able to almost completely block out the Red Team. The teams that were quick to correctly configure their routers and whitelists on their ESXi servers removed the largest holes in their network, and their service scores showed it. As a Red Team we really took advantage of Heartbleed and default pfSense credentials. Without those footholds, we weren't able to really do much. Smaller attack surfaces seemed to be a trend for a few of the CCDC regional events this year. My previous blog post talked about how at the Pacific Rim regional, the Red Team really only had 2 targets, and the vulnerabilities were default credentials. This was definitely not the case for South East, but the Red Team definitely noticed a lack of attack surface. I've been talking to a lot of people about some of these observations and we all agree that we want modern systems and network configurations, but how do you open up the attack surface without making it unrealistic?

All in all, I had an absolute blast at SECCDC. I'm already looking forward to next year. I know all the organizers of the SECCDC work incredibly hard to put on this event every year. Their efforts have been noticed and I thank them for all the time and effort they put forth to make this event a reality. And congratulations to UCF for winning a second year and a row! Good luck at Nationals and bring keep the championship in the South East!!

Pacific Rim Regional CCDC 2015 - Red Team

A week and a half ago I got to participate in the 2015 Pacific Rim (Pac-Rim) Collegiate Cyber Defense Competition as a member of the Red Team. My more experienced friend Dan has already written a couple of posts about this season's CCDC events (lockboxx) including Pac-Rim. I wanted to use this post not to talk about what I did as a member of the Red Team, but my developing opinions of CCDC and these types of "cyber defense" competitions.

To give context to this post, I'll give a brief description of Pac-Rim's scenario. The Blue Teams were tasked as the IT/Security department of The Center for Disease Control (CDC) while the world experienced a zombie outbreak. While trying the manage their network, they must also address the growing scare of zombies and how that impacted their jobs at the CDC.

As far as network design, the Blue Teams were given an external/public facing network, and a couple of internal networks with varying security levels. From the Red Team perspective we saw 3 primary targets, a VyOS router (VyOS), a Windows 2012 R2 exchange server, and a Fedora 20 web server. Initial scans for vulnerabilities turned up very little. These were pretty modern systems that weren't running a lot of services. Not a lot to attack. The VyOS router did have port forwarding rules set to proxy through connections to servers on the internal network. With this we could see a couple of services beyond the router such as MySQL. For the Red Team, the only way we actually got access to any of these machines was by default credentials.

Leaving default credentials is such a silly thing for Blue Teams to pass up and it is such an easy vector for Red Teams to leverage. Default credentials are usually the very first thing Blue Teams change. That being said, once default credentials have been changed, the Red Team has to be more cunning to find another way into the Blue Team's networks. Unfortunately, for the Pac-Rim event, it seemed that more than half of the teams separated themselves because they changed their passwords quicker than the other teams. I want to focus on this point a little bit more.

When I say that a Blue Team was quicker in changing their default password, I mean they were quicker by a minute, to seconds. This year, we split the Red Team up into cells. Each cell was tasked with attacking a specific Blue Team for the duration of the event. Each cell was to stay in sync with all the other Red Team cells. This sounds like a great concept because it seems very fair. And I agree that it is fair in theory. The issue you run into is the very opening few minutes. This entire event was determined by the first five minutes. The Blue Teams that changed their passwords on the Win2012 Exchange server before their router did better than the students that changed their router's password first. The reason being that the Win2012 Exchange server could be used to pivot all around the internal domain. The router did not provide this type of access. We even had a couple of teams that changed both the router and Exchange server passwords extremely quickly. The result, the Red Team really couldn't do much to them. Now, this isn't meant to be a Red Team sob story. I love when the students lock the Red Team out. That means they are learning, and they are equipped with the skills to make our industry safer. That to me is an amazing thing. The issue I have is that changing 2 default passwords and locking the Red Team out for a day and half is not a learning experience. Further more, since all the Red Team cells are staying in-sync with each other, A Blue Team could get away with leaving a gaping whole in their systems as long as the Red Team cells weren't attacking that particular issue at that time.

The primary point of CCDC is to provide the students with a unique learning experience that they will never get in the class room. At the end of the competition, I got to sit down with the Blue Team I attacked all weekend. They were so full of questions and eager to learn from their experience. The problem was, I couldn't answer all of their questions. Due to a scheduling issue, I had to work alone as a Red Team cell so my focus was extremely stretched. My Blue Team changed their Exchange server password right away and I never got access to it. I only got access to their router and MySQL database. Since most of the Blue Teams' networks was comprised of a majority of Windows systems, I was asked a lot of questions regarding how well they did configuring the workstations, domains, and other Windows related services. Unfortunately I had to tell them that because they changed 1 default password, I wasn't able to give them an accurate perspective. They could have had a terribly configured domain. I wouldn't know. And this was the case for a lot of Blue Teams. They simply didn't get all the feed back that the Red Team could have provided had there been more access.

I go back to the point that CCDC is suppose to be a learning experience. It's hard to find the balance between giving the students unrealistically insecure systems that the Red Team can stomp all over, and modern secure systems where the Red Team still has decent access. I also want to emphasize the point of these competitions is to focus on cyber security. One of my Blue Team members told me that they spent the entire first day (more than 8 hours) dealing with customer phone calls from the Orange Team. What?! Customer service has very little to do with developing cyber security skills. I understand that the Orange Team is there to act as real world customers, but this is a competition. Blue Teams are obviously going to put their best 'people person' on the phone. They probably won't learn anything in terms of how to deal with people and they'll miss out on the actual technical education.

In the end if my Blue Team was able to learn 1 thing, than in my opinion, the experience was worth it. I love doing these types of events as a way to connect with students and provide guidance in a way that it was provided to me. Next week I'll be participating in the South East CCDC regional and my company's own cyber defense competition (PCDC) so I should have a lot more stuff to report on.

GXPN Review: SANS SEC660 - Advanced Penetration Testing, Exploit Writing, and Ethical Hacking

So this blog update is incredibly overdue, but I guess better late than never. Back in August I was fortunate enough to be able to attend a session of the SANS Advanced Penetration Testing, Exploit Writing, and Ethical Hacking (SEC660) course. Overall, I can say that this course was an absolutely fantastic learning experience and a training event that I would highly recommend to anyone that is looking to further their penetration testing and exploit development skills.

Prior to taking 660, I took the SANS course Network Penetration Testing and Ethical Hacking (SEC560) back in January of 2013. Although I'm not going to write about that course in this post, 560 is another fantastic penetration testing course offered by SANS. My day-to-day job and personal interest has me focus more on the vulnerability and exploit development based research more so than the hands on penetration testing. As a result, a lot of the network based attack topics covered in this class were fairly new to me. Every day had so much content that the normal hours of the class were augmented with additional bootcamp labs. To make things even more interesting, I attended 660 during the SANS Virginia Beach conference which hosted their NetWars tournament. So I was spending pretty much every waking moment during the week hacking in some way, shape, or form.

The class is broken up into 6 days. Days 1-5 are content based learning days with frequent lab exercises. Day 6 wraps up the class with a capture the flag (CTF) challenge. Since there is so much content associated with this class, each day is fully packed. The instructor, Stephen Sims, did an absolutely amazing job at teaching this class. The following sections give a high level impression of each day. For more detailed topics covered during each day, I highly recommend checking out the course description on the SANS website.

Day 1 - Network Attacks for Penetration Testers

Day 1 was admittedly the most difficult for me. To start off the course, students are introduced to an extremely diverse set of advanced network attacks. Networking is one of my weaker areas, and as a result, I did need to spend a little more time completing some of the labs and studying this material. One of the best things I enjoyed about day 1 is the focus on obtaining access to a network that has security access controls. Many times, penetration testing descriptions start off with, 'assume you've been given initial access to the target network'. This is not always the case and it is extremely valuable to the customer when a penetration tester can bypass whatever network access controls may be in place.

Day 2 - Crypto, Network Booting Attacks, and Escaping Restricted Environments

Personally, I felt that Day 2 was the most unique content from course in the sense that the topics covered during this day are probably not areas immediately thought of when performing penetration tests. For example, cryptography. Often times as penetration testers, we'll look for a known implementation flaw online if we discover a particular crytographic algorithm being used, or defer the cryptanalysis to another time so that our testing event isn't consumed by trying to break cryptographic implementations. One of the take aways from this day is a methodology for efficiently evaluating a cryptographic implementation. Immediately with this skill set, your ability to provide a more comprehensive penetration test for you client greatly increases. Other notable topics covered in this class are delivering hypervisor boot images across a network (as a means to silently shim in between your victim's OS and hardware) and breaking out of kiosk style restricted environments.

Day 3 - Python, Scapy, and Fuzzing

This day of class was a lot of review for me. I've used Scapy for work projects, and I've been coding in Python for a few years now. I've also used a variety of fuzzers for both personal and work projects. The best part of this day, however, is getting some automagic scripts built by the SANS team for helping alleviate the pains that are associated with installing certain fuzzing platforms (I'm looking at you Sulley...). I love making things as simple as possible and having these new setup scripts eliminate the pains that come with installing some of these platforms is almost worth the entire cost of the class by itself ;).

Day 4 - Exploiting Linux for Penetration Testers

I really enjoyed days 4 and 5. I've worked through all the exercises in Hacking: The Art of Exploitation (ref) a couple of times and I've participated in a few CTFs requiring exploitation of Linux binaries. As a result, a lot of the information presented in this day was review for me. It was still a great lecture and lab day, however, because the best way to improve your exploit skills is to write exploits! I also find that relearning a topic taught by another knowledgeable instructor (in this case Stephen Sims) only helps solidify your understanding of a specific topic.

Day 5 - Exploiting Windows for Penetration Testers

Moving on from day 4, day 5 was all about Windows. Admittedly, I have a much greater understanding of low level Linux internals than I do Windows. That is something I'm currently working on. As a result of day 5, I came away with a much better understanding of the Windows linking and loading procedure and the in-memory process structure. Day 5 also focuses more on return-oriented programming (ROP) exploitation. To facilitate this, the class learned about the Immunity Debugger plugin, mona.py (source, manual), from the amazing Corelan team. Learning about this tool was the best part of Day 5 in my opinion.

Day 6 - Capture the Flag Challenge

After 5 intense days of instruction, it was time for us to put our new knowledge to use during the CTF event. Our class broke up into 4 teams of about 4 to 5 members per team. We had 6 uninterrupted competition hours. I focused mainly on the Linux exploitation challenges in the beginning and then moved on to Windows exploitation and a particularly tricky fuzzing challenge. After an absolutely thrilling competition, our team came out on top, taking home the SEC660 challenge coin!

Challenge Coin

Certification and Closing Thoughts

In November I successfully acquired my GXPN certification by taking the test associated with this class. I procrastinated studying much longer than I wanted to, but sometimes things come along that you can't plan for. In the end everything worked out. Additionally, I'd like to advocate this class for anyone seriously considering doing more exploit development in their career. Shortly after earning my GXPN, I was given an amazing job offer to be a full time exploit developer. I can honestly say that this class gave me the confidence and edge I needed during my interviews. Right now I'm currently transitioning into a fantastic opportunity within my current job so I didn't end up taking the new offer. Someday down the road, though, things might be different.

Overall I thought SEC660 was an absolutely amazing class and Stephen Sims did a fantastic job teaching. I'm already trying to plan out when I can take SEC760 - Advanced Exploit Development for Penetration Testers. 

Palmetto Cyber Defense Competition - 2014

Last weekend (April 12-13) was the second annual Palmetto Cyber Defense Competition (PCDC). Inspired by the national Collegiate Cyber Defense Competition (http://www.nationalccdc.org/), PCDC is a computer security and business competition open to schools in South Carolina. On Saturday, college teams compete while Sunday is reserved for high schools. Unlike CCDC competitions which usually span a couple of days, PCDC competitors are pressured into one fast and furious day of competition. I was fortunate enough to be a part of the Red team this year. I wanted to make this blog post to supplement the short Red team presentation given to the students at the end of their competition day. Since PCDC competition sessions are only a day long, there is so much to do in that timeframe. As a result, the Red team doesn't really get to talk with the students as much as we would like.

For this year's PCDC, we had a relatively small Red team. At any given time, we were between 7-8 members. As a result, we decided it would be most effective if we each took a team to focus on. If anyone of us would find a way into a system, we'd shout it out and each Red team member would try that method on their target Blue team. We wanted to keep things as even as possible. In addition to all of our technical tools, the Red team was allowed to use social engineering this year. The Red team had access to a VOIP phone which we could use to call the Blue teams. In addition, the Blue teams' competition area was in a large reception style room. This meant that the Red team could walk around the Blue teams to try and pick up what was going on.

From a social engineering perspective, the Red team kept things pretty simple. We stole a couple of Blue team 'mentor' name badges that would have been used to help identify individuals who could provide the Blue teams with some extra technical support. One of the defining differences with PCDC is its goal in encouraging STEM outreach. As a result, Blue team mentors work with the competing high school teams months in advance to help prepare them for the competition. As a result, the Blue teams grow to trust the mentors. Although I personally offered my Linux subject matter expertise to the Blue teams, I was not asked to help. I felt completely under-utilized. This is a positive note to point out. All the Blue teams for both college and high school were extremely cautious with allowing personal access to their space. As a result, very few of the Red team members got information via personal interaction. But we had other plans.

One of the Red team members brought a high quality camera with him. Equipped with nothing more than a collared shirt and good personality, he set out to film the Blue teams as a member of the 'local news station'. He was successful in taking a picture of every single Blue team's password lists for both college and high school days. Only one team reported his actions. As you can see by the following picture, the results were awesome:

Blue Team Passwords

In addition to pictures, we also tried to add our own business inject. Prior to the competition, one of our Red team members made a fake tax calculator program and accompanying inject form. The inject stated that the students were to report the error code given by the tax software as the upgrade from Windows XP to Windows 7 has caused some unanticipated errors. What really happened is the program would crash every single time but open up a reverse shell to our waiting machines. We had some success with this approach including one team that send us over 30 shells back! Other notable events include a team that redialed their phone immediately following a Blue team runner who called the Red team asking to clarify if the weird behavior on the team's system was the Red team or an actual technical issues. The Blue team runner was not a competitor, but one of the helpers there for the Blue teams. Because we are a friendly Red team and we want to help everyone out, we would always pick up our phone pretending to be the Gold team. As a result, this particular Blue team thought we were the Gold team for most of the day.

The Red team's game plan for attacking the Blue teams was simple. Get access, establish persistence, then have fun. This is true with pretty much every Red team for all the CCDC style events. I will say that for both college and high school days, every team was pretty good at keeping us out by changing their default passwords. The only way we got their default passwords was with our photographer. What the Blue teams weren't told was that the default passwords given to them were not all their default passwords. One of the services the Blue teams had to keep up was a Kunagi server. Kunagi is deployed on a Java application server like Tomcat or JBoss. For this event, the Blue teams were running JBoss. What every single one of them failed to notice, however, was that their JBoss admin console was publicly available and still had default admin credentials. This is how we got our initial access for every team.

Through the admin console we were able to upload our own WAR file (the deployable version of the Java applications that run on JBoss) which would start a reverse meterpreter session back to our machines. With this we would establish simple persistence by using meterpreter's run persistence module. From here, I no longer needed my custom deployed WAR, so I deleted is. This would hopefully hide the fact that I had compromised their system. With this initial access, the next step is to try and move laterally through the network. It turns out, these system were all part of a domain. So, our next step was to pull credentials from the machine. We did this with mimikatz. For those of you that don't know what mimikatz is, it is a tool that allows for plaintext recovery of cached Windows credentials. The following is a screenshot of what mimikatz output looks like:

Plaintext credentials with mimikatz

Now, lucky for the Red team (and consequently, unlucky for the Blue teams), these credentials belong to the domain administrator. From here, we were able to use the psexec command to log into the other Windows boxes and consequently establish persistence on all the Windows machines.

In addition to Windows, the Blue teams also had a couple of Linux servers and an ESXi server. As a team, we really didn't have too much luck on getting on the Linux servers except with a default SSH credential and through the ESXi console. But let's talk about how we got to the ESXi servers. For those of you who have used ESXi and connected through a browser, you know that it uses HTTPS. It turns out, ESXi is using OpenSSL. Hmmm. PCDC 2014 was on the weekend of April 12th and 13th. Not even a week before, the security community was thrown on its head with the HeartBleed vulnerability present in most modern versions of OpenSSL. On a hunch, we decided to try HeartBleed on the ESXi servers. We set up a simple script that would constantly ping the Blue team's ESXi servers and look for a HeartBleed response that contained a password field. About an hour later, one of our Red team members jumps up yelling, "it worked!" And there on his terminal, was a username and password field. It turns out that HeartBleed will work on an ESXi server and spit out usernames and passwords after a login attempt. Here is a screenshot of was our output looked like:

HeartBleed works on standard ESXi 5.5

One of the goals of PCDC is to give students a taste of the real world. How much better could be get than launching an attack against a critical service that exploits a vulnerability less than a week old? Needless to say this certainly caught some buzz with the spectators. Unfortunately, we did not discover the effectiveness of HeartBleed until the second day. As a result, the college students missed out on this. With these credentials we could point vSphere ESXi clients at the ESXi servers and help the Blue team's administer their VMs. My favorite is when I open up the console to a Linux machine and just see a # sitting there. For my Blue team, I was a little quicker on the keyboard than they were. I opened the console and saw the root prompt. I quickly changed the root password, turned on SSH, and then quit the user session through the vSphere client. This effectively locks the Blue team out of their root account and allows me to login remotely via SSH. My favorite result of this was having the Gold team call up the Red team with the response, "One of the Blue teams just said they saw their root password change right before their eyes and then lock them out. Was that you?"

At this point, it was time to start having fun with the students. A couple of the Red team member came very prepared for fun. In the year leading up to PCDC 2014, we have been hearing a lot about ransom-ware in the news. Software that gets on your computer, encrypts important files, and then demands a ransom for the decryption key. Inspired by this, one Red team member made some hilarity-ware. This combined the ransom-ware with some amusing side effects. Let me introduce Ponyware:

Ponies demand a ransom

Ponyware would encrypt a lot of the folders found on the computer and then insert a small ransom program in the front of every legitimate program it could find. The goal was to get the Blue teams to call the Red team for the decryption key after they paid a ransom with some of their points they had earned throughout the day. We successfully deployed Ponyware, but no Blue team called us for the decryption key. Instead, most of them either opted to leave it on and have it play its music, leave the desktop background a checkered board of ponies, and enjoy the custom mouse cursor. Others opted to take a large point deduction and request a Gold team re-image. Ironically, the re-image cost more points then the ransom would have.

Our next trick was pretty nasty. Originally, we were planning to deploy this uniformly at 3:30pm, but our plans didn't really work out. The plan was to overwrite all the master boot records (MBRs) on as many Blue team machines as possible. What we wrote the the MBRs was an awesome 512 byte program that displayed an animation on Nyan cat.

Nyan cat animation written to boot sector

Once the MBR was overwritten, we would bounce the boxes and this would be the only thing the Blue teams would see. Nyan cat was a huge hit with just about everyone. Although the Blue teams would have preferred not having their MBRs overwritten, they appreciated the humor. By the end of the day, a couple of computers were just sitting there completely useless counting down till 3:30pm:

Waiting until 3:30pm

From a Red team member's perspective, there are a couple of things I would have suggested to the Blue teams. In general, most teams did a very good job, but no matter how many times we say it, default passwords are going to kill you. As a Blue team member, you need to be aware of every service you have running and if/what those service's default passwords are. You must be aware of service that come out-of-the-box with default credentials. Now, I have only been a Red teamer on 2 CCDC style events, but both times I've bean able to access the ESXi servers with my vSphere client. In the case of PCDC, every Blue team was provided an adaptive security appliance (ASA). This can be used to restrict the IP addresses allowed to access the ESXi servers. Blue team's should also be aware of authentication mechanisms that are required and not required. The Blue team I focused on for day 2 only allowed SSH login via key authentication. This is a great move, except when the Gold team needs to be able to login via SSH to score your service and they don't have an authorized key because Gold team is using a password. Over and over again the Gold team remarked about how the Blue teams did not read their information packets thoroughly. The information packets usually have detailed information about your network and systems. As a result, you should know what is on your network. It shouldn't be towards the end of the day when I see this as I'm watching your session..

What is an ESXi?

At almost 1:30pm, the competition has been going on for about 5 hours. Just now trying to learn what an ESXi server is might mean you're a little behind the game. Another lesson learned (the hard way) by some Blue teams is if you know something bad is going to happen at a specific time (say 3:30pm), do not try and adjust your system clock past that time. Trying this trick quite literally blew up in some teams' faces.

I just want to note that some individuals on the Blue teams felt that we were a little hard on them. It was our intention as the Red team to not be complete jerks. Our goal is to educate the students while having fun with them. As we tried to tell the Blue teams, we do this because we care about education. So, if you're a current or future member of a Blue team, know that we don't do what we do to be mean. We do it to help you. If you're competing and the frustration is building, just remember, Red team loves you!

Red Team <3's you!

Red Team SE-CCDC 2014

Earlier this week I had the fantastic opportunity to participate in the South East regional Collegiate Cyber Defense Competition (CCDC). If you've never heard of CCDC before, here's the short and sweet. Teams of college students represent their schools by assuming the role of a Blue team which represents a business's IT branch. They are dropped into a preconfigured network after the previous IT employees were let go. They have no knowledge of what the systems, configurations, software, or even hardware look like before they get there. The Blue teams' goal is to ensure the daily operational aspects of the business remain up and operational. Here's the catch. The entire time they are trying to ensure their business stays online and available, the Red team (a team of security professionals playing the part of hackers) actively attack the Blue team's systems and try to take them offline. Not only must the Blue teams defend their networks, they must also complete a series of timed tasks administered by the judges (White team). The White teams gives the Blue teams business injects like opening up file sharing for the organization's employees or responding to potential incident response requests. Needless to say, the competition is a fast and furious exercise in information security and I was excited. Not only was this my first time as a CCDC Red team member, it was my first time at a CCDC period.

On Monday, a coworker and I drove out to this years location at Kennesaw State University. After the long drive, we setup in the Red team room and began scanning and enumeration. The rules for the first day were simple. We could scan the Blue teams all we wanted, but we were not allowed to perform any attacks. Our scans showed that there were a lot of systems and most of them looked pretty up to date. Every team had Windows 7 desktop boxes, a couple of Fedora 20  and CentOS Linux boxes, a Windows Server 2012 R2, and a couple of ESXi servers. By the end of the night, our best lead at an attack vector was an eCommerce site. I found a small SQL injection vulnerability in the search functionality and a couple other Red teamers found what looked to be a neat way of overwriting configurations by leveraging a misconfigured install directory. While we did our scanning, the Blue teams were hard at work configuring, administering, and hardening their infrastructure. To make things a little more interesting, the Red team worked with the competition's organizers (Gold team) to tip the scales a little bit more in our favor :).

Raphael Mudge (the creator of Armitage and Cobalt Strike - http://blog.strategiccyber.com/) worked with the Gold team to drop 2 of his Beacon backdoors on every team's network. This was used to simulate the insider threat left behind by the old IT employees that use to operate the networks that each Blue team inherited. Beacons act as an advanced persistence agent capable of 'beaconing' back on a timed interval. Each time Beacon checks back, it will query the Red team's attack server looking for a queue of commands to act on. These Beacons are heavily embedded on the Blue team's systems and have multiple fall back communication channels. In my opinion, it is one of Cobalt Strike's best features.

Sure enough, the next morning as soon  as the competition started, the Beacons started calling back. The Red team decided that the best way to play the game was to split up into pairs and every pair would focus on a team. Every time a pair found something with one team, we would yell it out the rest of the Red team to try on their target Blue team. I got to pair up with Mudge and get a first hand education on Cobalt Strike and collaborative Red teaming. Mudge and I had Team 5 (team Echo) who we affectionally referred to as our children for the next 2 days. We decided early on, like good parents, we would watch over our children very carefully. As soon as we got our first Beacon, we queued up the 'keylogger' command. Now, with every checkin, Beacon would dump a series of cached keystrokes. Almost immediately we got a sequence that looked like 'root [TAB] @D4#c.L8'. SWEET! Root creds! But to what?! I tried a couple of the Linux boxes but it didn't work. So I saved them for a little bit and moved on. A couple minutes later, another Red team member shared that they pulled the competition rules off another Blue team's box. The rules detailed all the default user names and passwords for the Blue team's systems. Good news for us! And it was still early in the day.

Using a nifty feature of Cobalt Strike, we attempted a mass login with default credentials. SUCCESS! We got a couple of successful logins including a Linux box. Mudge went to work securing our foothold on the Linux system. Thankfully for us, this was made easier by the default user being in the '/etc/sudoers' file. So a quick 'sudo /bin/sh' and we were good to go! A little later I remembered that there were ESXi servers on the Blue team's network so I popped open my vSphere client and pointed it at the ESXi server's IP address. Hmmm, usually the user name is root. Lets give our first sniffed password a try...SUCCESS again! A couple minutes later I logged into the second ESXi server with the default password 'changeme'! It was less than an hour into the competition on Day 2 and we had root on 2 ESXi servers, a Linux box, and 2 Windows boxes hooked. It looked like it was going to be a good day.

Root on both ESXi servers

To maintain my foothold on the servers, I added a new user (vm-admin) with full administrative privileges. Then I sat and watched. Slowly it looked like the Blue team was moving more of their assets to the ESXi servers. How nice of them! I decided that I wouldn't make myself known till the very end, after they had given me all of their VMs. This was a little risky. If they would find my user account, I could be booted from the server. Luckily for me, they didn't know I was there until I wanted them to. Between getting root and the end of the day though, we managed another big win for the Red team.

One of Metasploit's features in meterpreter is the ability to grab screen shots. Using Cobalt Strike, we configured the screen shot functionality to snag a capture every 10 seconds. This was a slow, sneaky way of spying on team Echo. Eventually, we saw a member of Echo open up Internet Explorer to administer a spiffy new pfSense box through the browser administration client. Mudge lit up when he saw this. It was time to test browser pivot in a real scenario! Cobalt Strike's browser pivot essentially allows you to spawn a process inside of a current Internet Explorer browser session and then proxy the session back to the Red team's local browser all unknowingly to the victim. This means that the browser has already authenticated and we can piggy back off of that. So now we can help administer the pfSense box for team Echo :).

Helping team 'Echo' administer their pfSense box via a browser pivot

With this capability we quickly configure pfSense to allow SSH and best yet, we could inject commands via the pfSense web admin console's 'execute' feature. So we added our SSH key for easy access. Later on, we would keylog the root password to the pfSense box as well. It was ours.

After all of this the end of the day was approaching quickly. At 6:00pm, the Blue teams must be hands off keyboards. Team Echo had a pretty solid run all through the day by keeping most of their services up and running. But, at 5:55pm I decided to change that. The first thing I did was change the root passwords on both ESXi servers. This is not enough to boot the root user off if they are logged in, however. So I saved the state of all the VMs and quickly bounced the physical servers. Team Echo would be kicked off and after the reboot, their passwords wouldn't be correct. It worked. To put the servers further out of reach I changed the passwords for all the users (1 server had 3 default users and the other had 4) and made sure SSH was disabled. The next morning they would have physical access to the box, but no way in. Now, I saved the state of the VMs for a reason. If I would have bounced the ESXi server without saving VM state, then the VMs would have to reboot and more than likely, require a valid login. But, if I saved their state and they were already logged in, then I wouldn't need to authenticate at all. Lucky for me, this was absolutely the case for a Linux and Windows box.

The next morning the competition's director came into the Red team room and asked which one of us owned team Echo's ESXi servers. Mudge and I gladly took credit. He asked us to reset the root passwords to both ESXi servers and give it back to team Echo. Since all of their services were on the servers, they literally couldn't do anything. Mudge and I agreed on one condition. We could backdoor the VMs we had access to. So after backdooring the Windows and Linux box and deleting team Echo's VM snapshots and replacing them with our own, we handed team Echo their new root passwords. Since today was a much shorter day, Mudge and I focused on trying to figure out why our Beacons stopped calling out to us. After a couple of hours we determined that team Echo had found the Beacons and removed them. Good for them, bad for us. About half an hour before the very end of the competition I attempted to login to the ESXi servers with my old account I created when I first got on the box. I only gave team Echo the root account passwords, nothing else. To my surprise, I still had complete admin access with my vm-admin account!

Since it was the end of the day, it was time to go all it. I took all permissions away from root, deleted the other users, deleted all the VMs and datastores, and dropped the servers into maintenance mode. Literally game over.

Here's the recap. From all that I just wrote, it would seem like team Echo might have been off their game. This is simply not true. I only described our successes. Not failures. Team Echo seemed to boot us off the Linux boxes every time we gained access. Every. Time. Our Beacons stopped calling back on both Windows boxes. Really the only things we had were the ESXi servers and the pfSense box. But guess what. We didn't even truly have the pfSense box. Team Echo knew we were there and they weren't even using it! Now, Mudge and I noticed a lack of activity with the firewall, but overall, a great move by team Echo. In the end, it was a couple of critical mistakes that cost team Echo.

Team Echo's first mistake was not changing the default password on the second ESXi server. Next, there was a lack of user auditing on the ESXi servers and the principle of least privilege was not enforced. All administrative/root accounts on a system should be documented and accounted for. If a new one shows up, say vm-admin, action needs to be taken. Team Echo still had root, so they should have immediately changed their password and deleted my account. By doing that, they would have effectively locked me out of the system. Finally, in a situation like CCDC, the Blue teams can't always prepare for curve balls like judges allowing Mudge to install Beacons on their systems, but they can protect against other things. An ESXi server should never be available outside of a local management network. This is were the pfSense box should have been stood up. A simple firewall rule to deny access from any IP address (except ScoreBot which is off limits to the Red team) would have prevented me from accessing the ESXi servers with my vSphere client even if I had the root password. And finally, never, ever leave default usernames and passwords on your system. The Red team WILL find them and they will take advantage of them.

All in all I had a lot of fun and learned a lot. Mudge was awesome to work with and I appreciate the time he took to help show me the ropes. In the next week or so I'll be ramping up for a local CCDC called Palmetto Cyber Defense Competition (PCDC). Hopefully I'll have more to share (I'll try to do more screen shots and less text) from that event shortly after.