Planet SysAdmin

September 05, 2015

Chris Siebenmann

Why we aren't tempted to use ACLs on our Unix machines

One of the things our users would really like to have is some easy way to do ad-hoc sharing of files with random collections of people. In theory this is a great fit for ACLs, since ACLs allow users themselves to extend various sorts of permissions to random people. Despite this appeal of ACLs, we have no interest in supporting them on our machines; in fact, we go somewhat out of our way to specifically block any chance that they might be available.

The core problem is that in practice today, ACL support is far from universal and not all versions of it behave the same way and are equally capable. What support you get (if any) depends on the OS, the filesystem, and if you're using NFS (as we are), what the NFS fileserver and its filesystem(s) support. As a practical matter, if we start offering ACLs we're pretty much committed to supporting them going forward, and to supporting a version of them that's fully backwards compatible with our initial version; otherwise users will likely get very angry with us for taking away or changing what will have become an important part of how they work.

(The best case on an ACL system change is that people would lose access to things that they should have access to, partly because people notice that right away. The worst case is that some additional people get access to things that they should not.)

Given that neither ACL support nor ACL behavior is anywhere near universal, a need for backwards compatibility is almost certain to limit our choice of future OSes, filesystems, and so on. Do we want to switch the fileservers to FreeBSD, for example, but NFS to ZFS on FreeBSD doesn't support the ACL semantics we need? We'd be out of luck and stuck. If we want the most future freedom we have to stick to the lowest common denominator, and today that is Unix UIDs, GIDs, and basic file permissions.

(This sort of future compatibility is not a universal need, of course. There are any number of environments out there where you build systems for specific needs and when those needs go away you're going to toss the systems. In that setup, ACLs today for one system don't necessarily imply ACLs tomorrow (or the same ACLs tomorrow) for another one.)

by cks at September 05, 2015 05:09 AM

September 04, 2015

Errata Security

About the systemd controversy...

As a troll, one of my favorite targets is "systemd", because it generates so much hate on both sides. For bystanders, I thought I'd explain what that is. To begin with, I'll give a little background.

An operating-system like Windows, Mac OS X, and Linux comes in two parts: a kernel and userspace. The kernel is the essential bit, though on the whole, most of the functionality is in userspace.

The word "Linux" technically only refers to the kernel itself. There are many optional userspaces that go with it. The most common is called BusyBox, a small bit of userspace functionality for the "Internet of Things" (home routers, TVs, fridges, and so on). The second most common is Android (the mobile phone system), with a Java-centric userspace on top of the Linux kernel. Finally, there are the many Linux distros for desktops/servers like RedHat Fedora and Ubuntu -- the ones that power most of the servers on the Internet. Most people think of Linux in terms of the distros, but in practice, they are a small percentage of the billions of BusyBox and Android devices out there.

The first major controversy in Linux was the use of what's known as the microkernel, an idea that removes most traditional kernel functionality and puts it in userspace instead. It was all the rage among academics in the early 1990s. Linus famously rejected the microkernel approach. Apple's Mac OS X was originally based on a microkernel, but they have since moved large bits of functionality back into the kernel, so it's no longer a microkernel. Likewise, Microsoft has moved a lot of functionality from userspace into the Windows kernel (such as font rendering), leading to important vulnerabilities that hackers can exploit. Academics still love microkernels today, but in the real world it's too slow.

The second major controversy in Linux is the relationship with the GNU project. The GNU project was created long before Linux in order to create a Unix-like operating system. They failed at creating a usable kernel, but produced a lot of userland code. Since most the key parts of the userland code in Linux distros comes from GNU, some insist on saying "GNU/Linux" instead of just "Linux". If you are thinking this sounds a bit childish, then yes, you are right.

Now we come to the systemd controversy. It started as a replacement for something called init. A running Linux system has about 20 different programs running in userspace. When the system boots up, it has only one, a program called "init". This program then launches all the remaining userspace programs.

This init system harks back to the original creation of Unix back in the 1970s, and is bit of a kludge. It worked fine back then when systems were small (when 640k of memory was enough for anybody), but works less well on today's huge systems. Moreover, the slight difference in init details among the different Linux distros, as well as other Unix systems like Mac OS X, *BSD, and Solaris, is a constant headache for those of us who have to sysadmin these boxes.

Systemd replaces the init kludge with a new design. It's a lot less kludgy. It runs the same across all Linux distros. It also boots the system a lot a faster.

But on the flip side, it destroys the original Unix way of doing things, becoming a lot more like how the Windows equivalent (svchost.exe) works. The Unix init system ran as a bunch of scripts, allowing any administrator to change the startup sequence by changing a bit of code. This makes understanding the init process a lot easier, because at any point you can read the code that makes something happen. Init was something that anybody could understand, whereas nobody can say for certain exactly how things are being started in systemd.

On top of that, the designers of systemd are a bunch of jerks. Linus handles Linux controversies with maturity. While he derides those who say "GNU/Linux", he doesn't insist that it's wrong. He responds to his critics largely by ignoring them. On the flip side, the systemd engineers can't understand how anybody can think that their baby is ugly, and vigorously defend it. Linux is a big-tent system that accepts people of differing opinions, systemd is a narrow-minded religion, kicking out apostates.

The biggest flaw of systemd is mission creep. It is slowly growing to take over more and more userspace functionality of the system. This complexity leads to problems.

One example is that it's replaced traditional logging with a new journal system. Traditional, text-based logs were "rotated" in order to prevent the disk from filling up. This could be done because each entry in a log was a single line of text, so tools could parse the log files in order to chop them up. The new journal system is binary, so it's not easy to parse, and hence, people don't rotate the logs. This causes the hard drive to fill up, killing the system. This is noticeable when doing things like trying to boot a Raspberry Pi from a 4-gigabyte microSD card. It works with older, pre-systemd versions of Linux, but will quickly die with systemd if something causes a lot of logging on the system.

Another example is D-Bus. This is the core system within systemd that allows different bits of userspace to talk to each other. But it's got problems. A demonstration of the D-Bus problem is the recent Jeep hack by researchers Charlie Miller and Chris Valasek. The root problem was that D-Bus was openly (without authentication) accessible from the Internet. Likewise, the "AllJoyn" system for the "Internet of Things" opens up D-Bus on the home network. D-Bus indeed simplifies communication within userspace, but its philosophy is to put all your eggs in one basket, then drop the basket.

Personally, I have no opinion on systemd. I hate everything. Init was an ugly kludge, and systemd appears to be just as ugly, albeit for difference reasons. But, the amount of hate on both sides is so large that it needs to be trolled. The thing I troll most about is that one day, "systemd will replace Linux". As systemd replaces more and more of Linux userspace, and begins to drive kernel development, I think this joke will one day become true.

by Robert Graham ( at September 04, 2015 08:54 PM

Why licensing wouldn't work

Would you allow an unlicensed doctor to operate on you? Many argue that cybersecurity professionals, and even software programmers, should be licensed by the government similar to doctors. The above question is the basis for their argument.

But this is bogus. The government isn't competent to judge doctors. It licenses a lot of bad doctors. It'll even give licenses to people who plainly aren't doctors. For example, in the state of Oregon, "naturopaths" (those practicing "natural", non-traditional medicine) can be licensed to be your primary care provider, prescribe medicines, and so on. Instead of guaranteeing "good" professionals, licensing gives an official seal of approval to "bad" practitioners. Naturopathy is, of course, complete nonsense, and Oregon politicians are a bunch of morons. (See the Portlandia series -- it's a documentary, not fiction).

Professions like licensing not because it improves the quality of the profession, but because it reduces competition. The steeper the licensing requirements, the more it keeps outsiders out. This allows the licensed to charge higher fees. This is why even bogus occupations like "hairdressers" seek licensing -- so they can charge more money.

Since different states license different occupations, we have nice experimental laboratory to measure the benefits of licensing. As the Wikipedia page on the subject documents, many have done the studies, and found no benefits.

Many argue for government to get involved in cybersecurity. Their premise is that government is a an oracle, all seeing and all wise. That's simply not true. Government can't figure out their own cybersecurity, so it's unreasonable to expect they can pass laws to help ours. Since they don't know cybersecurity, their solutions will be based on politics not reason. That's what their "CISA" bill attempts to solve cybersecurity with increased government surveillance -- because more surveillance is what government wants. This is why they punished North Korea based on flimsy evidence in the Sony attack, but ignored the hard evidence pointing to China in the GitHub attacks. Politically, beating up on North Korea is easy, but fighting China would entail unacceptable political costs.

As the Wassenaar cyber export rules demonstrated, government won't solve cybersecurity problems. It'll just create a whole new set of problems.

by Robert Graham ( at September 04, 2015 07:03 PM

Colin Percival

Tarsnap email confirmation bypass

Over the past four years, Tarsnap's bug bounties have received quite a bit of attention. Most of it has been very useful — almost 400 mistakes (most either cosmetic or harmless, but some of them significant) have been reported and fixed — but it does also get some unwanted attention: Despite my clear statement that Tarsnap's bug bounties are for problems in tarsnap code, not for problems in the website, I regularly see people running automated vulnerability scanners... which invariably yield a selection of absurd non-vulnerability "vulnerabilities".

One consequence of these unsolicited security scans is that — since they feed a variety of inputs to forms, including the account creation form — I see a lot of obviously fake signup attempts (alas, none yet from the world's most obviously fake domain name). These are harmless, since the signup code sends out a confirmation email and the account isn't actually created until the alleged registrant follows a link in that email; so I wasn't concerned when I received an email last week telling me that someone was trying to create an account as

Five minutes later, I was very concerned upon receiving an email telling me that the registration for had been confirmed and the account created.

September 04, 2015 08:00 AM

Chris Siebenmann

Consistency and durability in the context of filesystems

Here's something that I've seen trip people up more than once when they talk about filesystems. When we talk about what guarantees a filesystem provides to programs that write data to it, we can talk about two things and the difference between them can be important.

Durability is when you write something or change the filesystem and it's still there after the system crashes or loses power unexpectedly. Durability is what you need at a high level to say 'your email has been received' or 'your file has been saved'. As everyone hopefully knows, almost no filesystem provides durability by default for data that you write to files and many don't provide it for things like removing or renaming files.

What I'll call consistency is basically that the filesystem preserves the ordering of changes after a crash. If you wrote one thing then wrote a second thing and then had the system crash, you have consistency if the system will never wind up in a state where it still has the second thing but not the first. As everyone also hopefully knows, most filesystems do not provide data consistency by default; if you write data, they normally write bits of it to disk whenever they find it convenient without preserving your order. Some but not all filesystems provide metadata consistency by default.

(Note that metadata consistency without data consistency can give you odd results that make you unhappy. Consider 'create new file A, write data to A, remove old file B'; with metadata consistency and no data consistency or forced durability, you can wind up with an empty new file A and no file B.)

Durability and consistency are connected but one does not necessarily require the other except in the extreme case of total durability (which necessarily implies total consistency). In particular, it's entirely possible to have a filesystem that has total consistency but no durability at all. Such a filesystem may rewind time underneath applications after a crash, but it will never present you with an impossible situation that didn't exist at some pre-crash point; in the 'write A, write B, crash' case, you may wind up with nothing, A only, or A and B, but you will never wind up with just B and no A.

(Because of its performance impact, most filesystems do not make selective durability of portions of the filesystem impose any sort of consistency outside of those portions. In other words, if you force-flush some files in some order, you're guaranteed that your changes to those files will have consistency but there's no consistency between them and other things going on.)

Applications not infrequently use forced flushes to create either or both of durability (the DB committed the data it told you it did) and consistency (the DB's write log reflects all changes in the DB data files because it was flushed first). In some environments, turning off durability but retaining or creating consistency is an acceptable tradeoff for speed.

(And some environments don't care about either, because the fix procedure in the face of an extremely rare system crash is 'delete everything and restart from scratch'.)

Note that journaled filesystems always maintain consistent internal data structures but do not necessarily guarantee that consistency for what you see, even for metadata operations. A journaled filesystem will not explode because of a crash but it may still partially apply your file creations, renames, deletions and so on out of order (or at least out of what you consider order). However it's reasonably common for journaled filesystems to have fully consistent metadata operations, partly because that's usually the easiest approach.

(This has some consequences for developers, along the same lines as the SSD problem but more so since it's generally hard to test against system crashes or spot oversights.)

by cks at September 04, 2015 05:11 AM

September 03, 2015

Errata Security

Review: Rick and Morty

The best sci-fi on television right now is an animated series called Rick and Morty on the Cartoon Network.

You might dismiss it, as on the surface it appears to be yet another edgy, poorly-drawn cartoon like The Simpsons or South Park. And in many ways, it is. But at the same time, hard sci-fi concepts infuse each episode. Sometimes, it's a parody of well-known sci-fi, such as shrinking a ship to voyage through a body. In other cases, it's wholly original sci-fi, such as creating a parallel "micro" universe whose inhabitants power your car battery. At least I think it's original. It might be based on some obscure sci-fi story I haven't read. Also, the car battery episode is vaguely similar to William Gibson's latest cyberpunk book "The Peripheral".

My point is this. It's got that offensive South Park quality that I love, but mostly, what I really like about the series is its hard sci-fi stories, and the way it either parodies or laughs at them. I know that in next year's "Mad Kitties" slate, I'm definitely going to write in Rick and Morty for a Hugo Award.

by Robert Graham ( at September 03, 2015 11:40 PM

Everything Sysadmin

CfP: USENIX Container Management Summit (UCMS '15)

The 2015 USENIX Container Management Summit (UCMS '15) will take place November 9, 2015, during LISA15 in Washington, D.C.

Important Dates

  • Submissions due: September 5, 2015, 11:59 p.m. PDT
  • Notification to participants: September 19, 2015
  • Program announced: Late September 2015

(quoting the press release):

UCMS '15 is looking for relevant and engaging speakers and workshop facilitators for our event on November 9, 2015, in Washington, D.C. UCMS brings together people from all areas of containerization--system administrators, developers, managers, and others--to identify and help the community learn how to effectively use containers.

Submissions Proposals may be 45- or 90-minute formal presentations, panel discussions, or open workshops.

This will be a one-day summit. Speakers should be prepared for interactive sessions with the audience. Workshop facilitators should be ready to challenge the status quo and provide real-world examples and strategies to help attendees walk away with tools and ideas to improve their professional lives. Presentations should stimulate healthy discussion among the summit participants.

Submissions in the form of a brief proposal are welcome though September 5, 2015. Please submit your proposal via email to You can also reach the chairs via that email address with any questions or comments. Presentation details will be communicated to the presenters of accepted talks and workshops by September 19, 2015. Speakers will receive a discount for the conference admission. If you have special circumstances, please contact the USENIX office at

Click for more info.

September 03, 2015 06:00 PM

CfP: USENIX Release Engineering Summit (URES '15)

Hey all you devops, CI/CD/CD people! Hey all you packagers, launchers, and shippers. Hey all your containers mavins and site reliability engineers!

Submissions due: September 4, 2015 - 11:59 pm

(quoting the press release):

At the third USENIX Release Engineering Summit (URES '15), members of the release engineering community will come together to advance the state of release engineering, discuss its problems and solutions, and provide a forum for communication for members of this quickly growing field. We are excited that this year LISA attendees will be able to drop in on talks so we expect a large audience.

URES '15 is looking for relevant and engaging speakers for our event on November 13, 2015, in Washington, D.C. URES brings together people from all areas of release engineering--release engineers, developers, managers, site reliability engineers and others--to identify and help propose solutions for the most difficult problems in release engineering today.

Click for more info.

September 03, 2015 04:59 AM

Chris Siebenmann

How I've decided to coordinate multiple git repos for a single project

I'm increasingly using git for my own projects (partly because I keep putting them on Github), and this has brought up a problem. On the one hand, I like linear VCS histories (even if they're lies); I don't plan on having branches be visible in the history of my own repos unless it's clearly necessary. On the other hand, I routinely have multiple copies of my repos spread across multiple machines. In theory I always keep all repos synchronized with each other before I start working in one and make commits. In practice, well, not necessarily, and the moment I screw that up a straightforward git pull/push workflow to propagate changes around creates merges.

My current solution goes like this. First, I elect one repo as the primary repo; this is the repo which I use to push changes to Github, for example. To avoid merge commits ever appearing in it, I set it to only allow fast-forward merges when I do 'git pull', with:

git config pull.ff only

This insures that if the primary repo and a secondary repo wind up with different changes, a pull from the secondary into the primary will fail instead of throwing me into creating a merge commit that I don't want. To avoid creating merge commits when I pull the primary into secondaries, all other repos are set to rebase on pulls following my standard recipe. This is exactly what I want; if I pull new changes from the primary into a secondary, any changes in the secondary are rebased on top of the primary's stuff and linear history is preserved. I can then turn around and pull the secondary's additional changes back into the primary as a fast-forward.

If I use 'git push' to move commits from one repo to another I'm already safe by default, because git push normally refuses to do anything except fast-forward updates of the remote. If it complains, the secondary repo involved needs a rebase. I can either do the rebase with 'git pull' in the secondary repo, or in the primary repo I can push to the remote tracking branch in the secondary with 'git push <machine>:<directory> master:origin/master' and then do a 'git rebase' on the secondary.

(Using a push from the primary usually means that my ssh activity flows the right way. And if I'm pushing frequently I should configure a remote for the secondary or something. I'm not quite hep on git repo remotes and remote tracking branches just yet, though, so that's going to take a bit of fumbling around when I get to it.)

by cks at September 03, 2015 04:56 AM

toolsmith #108: Visualizing Network Data with Network Data


R development environment (R, RStudio)

This month finds us in a new phase for toolsmith as it will not be associated with ISSA or the ISSA Journal any further. Suffice it to say that the ISSA board and management organization decided they no longer wanted to pay the small monthly stipend I’d been receiving since the inception of the toolsmith column. As I am by no means a profiteer, I am also not a charity, so we simply parted ways. All the better I say, as I have been less than satisfied with ISSA as an organization: Ira Winkler and Mary AnnDavidson should serve to define that dissatisfaction.
I will say this, however. All dissatisfaction aside, it has been my distinct pleasure to write for the ISSA Journal editor, Thom Barrie, who has been a loyal, dedicated, committed, and capable editor and someone I consider a friend. I will miss our monthly banter, I will miss him, and I thank him most sincerely for these nine years as editor. The ISSA Journal is better for his care and attention. Thank you, Thom.
Enough said, what’s next? I’ll continue posting toolsmith here while I consider options for a new home or partnership. I may just stick exclusively to my blog and see if there is a sponsor or two who might be interested in helping me carry the toolsmith message.
I thought I'd use our new circumstances to test a few different ideas with you over the next few months, your feedback is welcome as always, including ideas regarding what you might like to see us try. As always toolsmith will continue to offers insights on tools useful to the information security practitioner, typically open source and free.

To that end, I thought I'd offer you a bit of R code I recently cranked out for a MOOC I was taking. The following visualizations with R are the result of fulfilling a recent assignment for Coursera’s online Data Visualization class. The assignment was meant to give the opportunity to do non-coordinate data visualization with network data as it lends itself easily to graph visualization. I chose, with a bit of cheekiness in mind, to visualize network data…wait for it…with security-related network data.

Data Overview

I gathered data for the assignment from a network traffic packet capture specific to malware called Win32/Sirefef or ZeroAccess that uses stealth to hide its presence on victim systems. This Trojan family runs the gamut of expected behaviors, including downloading and running additional binaries, contacting C2, and disabling system security features. The Microsoft Malware Protection Center reference is here.
The packet capture I used was gathered during a ZeroAccess run-time analysis in my lab using a virtualized Windows victim and Wireshark, which allowed me to capture data to be saved as a CSV. The resulting CSV provides an excellent sample set inclusive of nodes and edges useful for network visualization. Keep in mind that this is a small example with a reduced node count to avoid clutter and serve as an exemplar. A few notes about the capture:
  • Where the protocol utilized was HTTP, the resulting packet length was approximately 220 bytes.
  • Where the protocol was TCP other than HTTP, the resulting packet length was approximately 60 bytes.
  • For tidy visualization these approximations are utilized rather than actual packet length.
  • Only some hosts utilized HTTP, specific edges are visualized where appropriate.
A summary of the data is available for your review after the Graphviz plots at the end of this document.

DiagrammeR and Graphviz

The DiagrammeR package for R includes Graphviz, which, in turn, includes four rendering engines including dot, neato, twopi, and circo. I’ve mentioned Graphviz as part of my discussion of ProcDot and AfterGlow as it is inherent to both projects. The following plots represent a subset of the ZeroAccess malware network traffic data.
- The green node represents the victim system.
- Red nodes represent the attacker systems.
- Orange nodes represent the protocol utilized.
- The cyan node represent the length of the packet (approximate.)
- Black edges represent the network traffic to and from the victim and attackers.
- Orange edges represent hosts conversing over TCP protocol other than HTTP.
- Cyan edges represent the relationship of protocol to packet length.
- Purple edges represent hosts communicating via the HTTP protocol.
Graphs are plotted in order of my preference for effective visualization; code for each follows.

After these first four visualizations, keep reading, I pulled together a way to read in the related CSV and render a network graph automagically.

Visualization 1: Graphviz ZeroAccess network circo plot

Visualization 1 code

digraph {

graph [overlap = false]

node [shape = circle,
style = filled,
color = black,
label = '']

node [fillcolor = green]
a [label = '']

node [fillcolor = red]
b [label = '']
c [label = '']
d [label = '']
e [label = '']
f [label = '']

node [fillcolor = orange]
g [label = 'TCP']
h [label = 'HTTP']

node [fillcolor = cyan]
i [label = '60']
j [label = '220']

edge [color = black]
a -> {b c d e f}
b -> a
c -> a
d -> a
e -> a
f -> a

edge [color = orange]
g -> {a b c d e f}

edge [color = purple]
h -> {a b}

edge [color = cyan]
g -> i
h -> j
engine = "circo")


Visualization 2: Graphviz ZeroAccess network dot plot

Visualization 2 code

digraph {

graph [overlap = false]

node [shape = circle,
style = filled,
color = black,
label = '']

node [fillcolor = green]
a [label = '']

node [fillcolor = red]
b [label = '']
c [label = '']
d [label = '']
e [label = '']
f [label = '']

node [fillcolor = orange]
g [label = 'TCP']
h [label = 'HTTP']

node [fillcolor = cyan]
i [label = '60']
j [label = '220']

edge [color = black]
a -> {b c d e f}
b -> a
c -> a
d -> a
e -> a
f -> a

edge [color = orange]
g -> {a b c d e f}

edge [color = purple]
h -> {a b}

edge [color = cyan]
g -> i
h -> j
engine = "dot")
Visualization 3: Graphviz ZeroAccess network twopi plot

Visualization 3 code
digraph {

graph [overlap = false]

node [shape = circle,
style = filled,
color = black,
label = '']

node [fillcolor = green]
a [label = '']

node [fillcolor = red]
b [label = '']
c [label = '']
d [label = '']
e [label = '']
f [label = '']

node [fillcolor = orange]
g [label = 'TCP']
h [label = 'HTTP']

node [fillcolor = cyan]
i [label = '60']
j [label = '220']

edge [color = black]
a -> {b c d e f}
b -> a
c -> a
d -> a
e -> a
f -> a

edge [color = orange]
g -> {a b c d e f}

edge [color = purple]
h -> {a b}

edge [color = cyan]
g -> i
h -> j
engine = "twopi")


Visualization 4: Graphviz ZeroAccess network neato plot

Visualization 4 code

digraph {

graph [overlap = false]

node [shape = circle,
style = filled,
color = black,
label = '']

node [fillcolor = green]
a [label = '']

node [fillcolor = red]
b [label = '']
c [label = '']
d [label = '']
e [label = '']
f [label = '']

node [fillcolor = orange]
g [label = 'TCP']
h [label = 'HTTP']

node [fillcolor = cyan]
i [label = '60']
j [label = '220']

edge [color = black]
a -> {b c d e f}
b -> a
c -> a
d -> a
e -> a
f -> a

edge [color = orange]
g -> {a b c d e f}

edge [color = purple]
h -> {a b}

edge [color = cyan]
g -> i
h -> j
engine = "neato")

Read in a CSV and render plot

Populating graphs arbitrarily as above as examples is nice...for examples. In the real world, you'd likely just want to read in a CSV derived from a Wireshark capture.
As my code is crap at this time, I reduced zeroaccess.csv to just the source and destination columns, I'll incorporate additional data points later. To use this from your own data, reduce CSV columns down to source and destination only.
Code first, with comments to explain, derived directly from Rich Iannone's DiagrammerR example for using data frames to define Graphviz graphs.

Visualization 5 is your result. As you can see, is the center of attention and obviously our ZeroAccess victim. Yay, visualization!

Visualization 5

Following is a quick data summary, but you can grab it from Github too.

Network Data

Summary: zeroaccess.csv

zeroaccess <- span=""> read.csv("zeroaccess.csv", sep = ",")

##             Source            Destination  Protocol       Length       
## HTTP: 36 Min. : 54.00
## : 90 : 90 TCP :456 1st Qu.: 60.00
## 6 6 Median : 62.00
## : 6 : 6 Mean : 84.98
## : 6 : 6 3rd Qu.: 62.00
## : 3 : 3 Max. :1506.00
## (Other) : 41 (Other) :229


##           Source    Destination Protocol Length
## 1 TCP 62
## 2 TCP 62
## 3 TCP 62
## 4 TCP 62
## 5 TCP 54
## 6 HTTP 221

In closing

Hopefully this leads you to wanting to explore visualization of security data a bit further, note the reference material in Acknowledgments.
I've stuffed all this material on Github for you as well and will keep working on the CSV import version as well.
Ping me via email or Twitter if you have questions (russ at holisticinfosec dot org or @holisticinfosec). Cheers…until next month.


Rich Iannone for DiagrammeR and the using-data-frames-to-define-graphviz-graphs example
Jay and Bob for Data-Driven Security (the security data scientist's bible)

by Russ McRee ( at September 03, 2015 03:41 AM

September 02, 2015

Colin Percival

Safe from what?

I woke up this morning to a headline news story on CBC's website: Is your baby monitor safe? According to a literal reading of Betteridge's law of headlines, the answer to this question should be "no", although if you consider the spirit of the law — as a commentary on sensationalist journalism — then the answer should probably be "yes". To me, neither answer makes sense, because the question itself doesn't make sense.

September 02, 2015 11:30 PM

LZone - Sysadmin

Building a Generic Sysadmin Policy Scanner

After writing the same scripts several times I decided it is time for a generic solution to check Debian servers for configuration consistency. As incidents and mistakes happen each organization collects a set of learnings (let's call it policies) that should be followed in the future. And one important truth is that the free automation and CM tools we use (Chef, Puppet, Ansible, cfengine, Saltstack...) allow to implement policies, but do not seem to care much about proofing correct automation.

How to ensure following policies?

But how to really ensure following these policies? The only way is by checking them and revisiting the check results frequently. One could build a script and send a daily/weekly mail report. This is always a custom solution and that's what I did several times already. So I do it one final time, but this times in a generic way.

Generic Policy Scanning

For me a generic configuration consistency / policy scanner has at least the following requirements:
  1. Optional generic pre-defined policies
  2. Optional custom user-defined policies
  3. Policies checked locally on the host
  4. Policies checked from CM systems
  5. Per host/hostgroup policy enabling
  6. Generic discovery of your hosts
  7. Dynamic per policy/group/host result filtering
  8. Customizable mail reports
  9. Result archival for audits
  10. Some simple trending
  11. Daily diffs, New findings, Resolved Isses
  12. Acknowledging Findings
I started implementing a simple solution (entirely bash and SSH based, realizing requirements 1,2,3,4,6,7,9,10) with It is quite easy to setup by configuring the type of
  • Host list provider (e.g. Chef, Puppet, mcollective)
  • SSH access available
  • Sudo yes/no
and you can run it instantly with the default set of policy scanners (which of course not necessarily all make sense for all type of systems).

Implemented Scanners

By setting up the results and the static HTML (instructions in in some webserver document root you can browse through the results.


Result overview:

Filter details:

September 02, 2015 07:22 PM

Sean's IT Blog

EUC5404 – Deliver High Performance Desktops with VMware Horizon and NVIDIA GRID vGPU

Notes from EUC5405.

Reasons for 3D Graphics

  • Distributed Workforces with Large Datasets – harder to share
  • Contractors/3rd Party workers that need revocable access – worried about data Leakage and Corporate Security

Engineering firm gained 70% productivity improvements for CATIA users by implementing VDI – slide only shows 20%

Windows 7 drives 3D graphics, Aero needs 3D.  Newer versions of Windows and new web browsers do even more.

History of 3D Graphics in Horizon

  • Soft3D was first
  • vSGA – shared a graphics card amongst VM, limited to productivity and lightweight use
  • vDGA – hardwire card to virtual machine
  • GRID vGPU – Mediated Pass-thru, covers the middle space between vSGA and vDGA

vGPU defined – Shared access to physical GPU on a GRID card, gets access to native NVIDIA drivers

vGPU has official support statements from application vendors

Product Announcement – 3D graphics on RDSH

vGPU does not support vMotion, but it does support HA and DRS placement

Upgrade Path to Horizon vGPU

If you already have GRID cards and are using vDGA or vSGA, there is an upgrade path to vGPU.


  • Upgrade to vSphere 6.0
  • Upgrade Horizon to 6.1 or newer
  • Install NVIDIA VIBs on host
  • Upgrade VMs to version 11
  • Set vGPU profiles
  • Install drivers in VMs

vGPU has Composer Support

GRID Profiles set in vCenter

Two settings to configure – one in vCenter (vGPU Profiles) and one in Horizon

GRID 2.0 – bringing Maxwell to GRID

More users, Linux Support

Moving to Platform – software on top of hardware instead of dedicated product line for GRID

GRID 2.0 is hardware plus software.  Changing from being a driver into a platform and software with additional features

Licensing is changing. Licensed user groups.

Grid Editions

vMotion not coming today – much more complicated problem to solve

GRID editions

GRID Use Cases

Virtual PC – business users who expect great perf, AutoCAD, PhotoShop

Virtual Workstation – Siemens, Solidworks, CATIA, REVIT

Virtual Workstation Extended – Very high end.  Autodesk Maya


High-Perf VDI is not the same your regular VDI

  • Density goes down, CPU/Memory/IOPS/Rich Graphics capabilities go up
  • Workloads are different than traditional VDI

Hardware Recommendations

  • vSphere 6.0 Required
  • VM must be HW version 11
  • 2-8 vCPUs, at least 4 for Power Users
  • Minimum 4GB RAM
  • 64-bit OS

Required Components in VMs:

  • VM Tools
  • View Agent
  • NVIDIA Driver

Use the VMware OS Optimization Tool fling.  Users can see up to 40% in resource savings.

Sizing Rich Graphics – Storage

Storage still critical factor in performance

CAD users can demand more than 1TB of storage per desktop

Size and performance matter now

Storage Options:

  • Virtual SAN – SSD based local storage
  • Or All-Flash based SANs

Bringing Rich 3D into Production

  • Establish End-User Acceptance Criteria to verify that User Experience is acceptable
  • Have end users test applications and daily tasks
  • Time how long it takes to complete tasks

by seanpmassey at September 02, 2015 05:36 PM

VAPP5483 – Virtualizing Active Directory the Right Way

Notes from VAPP5483 – Virtualizing Active Directory the Right Way

Active Directory Overview

Windows Active Directory multi-master replication conundrum

Writes originate from any DC

Changes must converge

  • Eventually
  • preferably on time

Why virtualize Active Directory

  • Virtualization is mainstream at this point
  • Active Directory is fully supported in virtual environments
  • Active Directory is virtualization friendly -> Distributed multi-master model, low resource requirements
  • Domain Controllers are interchangable -> one breaks, they can be replaced. Cattle, not pets
  • Physical domain controllers waste compute resources

Common Objections to DC Virtualization

  • Fear of the stolen VMDK -> no different than stolen server or backup tape
  • Priviledge Escalation -> vCenter priviledges are separate
  • Have to keep certain roles physical -> no technical reason for this, can seize or move roles if needed
  • Deviates from standards/build process -> helps standardization
  • Time Keeping in VMs is hard -> Presenters agree

Time Sync Issues

Old way – VMs get time from ESXi

Changed to use Windows time tools

KB 1189 -> time sync with host still happens on vMotion or Guest OS reboot

Demo -> moving PDC emulator to host with bad clock

If time on host is more than 1 year old, NTP cannot update or fix the time

How do we determine the correct time

Ask ESXi host?

This could be OK if…

  • Host times are always right
  • CMOS doesn’t go bad
  • Rogue operations don’t happen
  • Security is a thing other people worry about

Reality – Stuff happens…

vSphere default behavior corrects time on the PDC emulator

Can cause a lot of issues in impacted Windows Forests

Preventing Bad Time Sync

  • Ensure hardware clock is correct
  • Configure reliable NTP
  • Disable DRS on PDCe
  • Use Host-Guest Affinity for PDCes
  • Advanced Settings to disable Time Sync –> KB 1189

Best Practices

Don’t use WAN for Auth –  Place domain controllers locally

Distribute FSMO Roles

Use Effective RBAC – don’t cross roles unless needed, give rights only to trusted operators

To P2V or Not – don’t do it unless you hate yourself

Use Anti-Affinity Rules -> don’t have DCs on the same hosts, use host rules to place important


vCPU – under 10K users, 1 vCPU, over that, start with 2 vCPU

RAM – database server, database is held in RAM, more RAM is better, perfmon counter shows cache usage

Networking – VMXNET3

Storage – Space that it needs plus room to grow


70% of issues are DNS issues

AD requires effective DNS

DNS solution – doesn’t matter if Windows or Appliance, but must be AD-Aware

Avoid pointing DNS to itself, otherwise DNS cannot start

Virtual Disk -> Caching MS KB 888794

Preventing USN Rollback

AD is distributed directory service, relies on clock-based replication

Each DC keeps track of all transactions and tags them with a GUID

If a DC is snapshotted and rolled back, local DC will believe it is right, but all others will know it is bad and refuse to replicate with it. This is called USN rollback

Demo USN rollback

If you have 2008 R2 and below DCs, they will stop replicating. Both will still advertise as domain controllers

VM-Generation ID – exposes counter to guest

  • 2012 and newer. Operating system level feature and must be supported by hypervisor
  • vSphere 5.0 Update 2 and newer
  • Attribute is tracked in local copy of database on local domain controller, triggered by snapshots and snapshot rollback

Provides protection against USN rollback

Invented specifically for virtual domain controllers, allows for cloning of domain controllers

Demo – Clone a Domain Controller

Domain Controller must have software and services that support cloning – agents have to support cloning

Do NOT hot clone a domain controller. Must be in powered off state

Do not clone a DC that holds FSMO roles

Can Clone the PDCe, must power up reference domain controller before powering on clone

DNS must work

Do not sysprep the system

DC Safeguard allows a DC that has been reverted/restored to function as a DC

How it works:

  • VM Generation ID checked on DC boot, when a snapshot is created,  or when the VM is reverted to an old snapshot.  VM Generation-ID on VM is checked against the copy in the local database.
  • If it differs, RID Pool dumped and new RID pool issued
  • When Generation ID has changed, AD will detect it and remediate it
  • RID pool discarded, get new RID Pool and objects are re-replicated. VM essentially becomes a new DC

by seanpmassey at September 02, 2015 05:15 PM

Racker Hacker

Impostor syndrome talk: FAQs and follow-ups

I’ve had a great time talking to people about my “Be an inspiration, not an impostor” talk that I delivered in August. I spoke to audiences at Fedora Flock 2015, Texas Linux Fest, and at Rackspace. The biggest lesson I learned is that delivering talks is exhausting!

Frequently Asked Questions

Someone asked a good one at Fedora Flock:

How do you deal with situations where you are an impostor for a reason you can’t change? For example, if you’re the only woman in a male group or you’re the youngest person in a mostly older group?

I touched on this a bit in the presentation, but it’s a great question. This is one of those times where you have to persevere and overcome the things you can’t change by improving in all of the areas where you can change.

For example, if you’re the youngest in the group, find ways to relate to the older group. Find out what they value and what they don’t. If they prefer communication in person over electronic methods, change your communication style and medium. However, you shouldn’t have to change your complete identity just for the rest of the group. Just make an adjustment so that you get the right response.

Also, impostor syndrome isn’t restricted to a particular gender or age group. I’ve seen it in both men and women in equal amounts, and I’ve even seen it in people with 40 years of deep experience. It affects us all from time to time, and we need structured frameworks (like OODA) to fight it.

How do I battle impostor syndrome without becoming cocky and overconfident?

The opposite of impostor syndrome, often called the Dunning-Kruger Effect, is just as dangerous. Go back the observe and orient steps of the OODA loop (see the slides toward the end of the presentation) to be sure that you’re getting good feedback from your peers and leaders. Back up your assertions with facts and solid reasoning to avoid cognitive bias. Bounce those ideas and assertions off the people you trust.

When I make an assertion or try to get someone else to change what they’re doing, I’ll often end with “Am I off-base here?” or “Let me know if I’m on the right track” to give others an opportunity to provide criticism. The added benefit is that these phrases could drag someone with impostor syndrome out of the shadows and into the discussion.

That leads into another good question I received:

How can we reduce impostor syndrome in open source communities as a whole?

The key here is to find ways to get people involved, and then get them more involved over time. If someone is interested in participating but they aren’t sure how to start, come up with ways they can get involved in less-formal ways. This could be through bug triaging, fixing simple bugs, writing documentation, or simply joining some IRC meetings. I’ve seen several communities go through a process of tagging bugs with “easy” tags so that beginners can try to fix them.

Another more direct option is to call upon people to do certain things in the community and assign them a mentor to help them do it. If someone isn’t talking during an IRC meeting or piping up on a mailing list, call them out — gently. It could be something as simple as: “Hey, [name], we know you’re knowledgeable in [topic]. Do you think this is a good idea?” Do that a few times and you’ll find their confidence to participate will rise quickly.


Insides vs. outsides

Someone stopped me outside the talk room at Texas Linux Fest and said a leader at his church summarized impostor syndrome as “comparing your insides to someone else’s outsides”. That led me to do some thinking.

Each and every one of us has strengths and weaknesses. I’d wager that we all have at least once vice (I have plenty), and there are things about ourselves that we don’t like. Everyone has insecurities about something in their life, whether it’s personal or professional. These are things we can’t see from looking at someone on the outside. We’re taking our laundry list of issues and comparing it to something we think is close to perfection.

Don’t do that. It’s on my last slide in the presentation.

You know at least one thing someone else wants to know

After doing the talk at Rackspace, I was pulled into quite a few hallway conversations and I received feedback about my presentation. In addition, many people talked about their desire to get up and do a talk, too. What I heard most often was: “I want to do a talk, but I don’t know what to talk about.”

It reminds me of a post I wrote about writing technical blogs. There is at least one thing you know that someone else wants to know. You might be surprised that the most hit post on my blog is an old one about deleting an iptables rule. Deleting an iptables rule is an extremely basic step in system administration but it’s tough to remember how to do it if you don’t use the iptables syntax regularly.

Rackspace holds Tech Talk Tuesdays during lunch at our headquarters in San Antonio each week. It’s open to Rackers and escorted guests only for now, but our topic list is wide open. Rackers have talked about highly technical topics and they’ve also talked about how to brew beer. I’ve encouraged my coworkers to think about something within their domain of expertise and deliver a talk on that topic.

Talk about your qualifications and experience without bragging

You can be humble and talk about your strengths at the same time. They aren’t mutually exclusive. It can be a challenge to bring these things up during social settings, especially job interviews. My strategy is to weave these aspects about myself into a story. Humans love stories.

As an example, if you’re asked about your experience with Linux, tell a short story about a troubleshooting issue from your past and how you solved it. If you’re asked about your python development experience, talk about a project you created or a hard problem you solved in someone else’s project. Through the story, talk about your thought process when you were solving the problem. Try your best to keep it brief. These stories will keep the other people in the room interested and it won’t come off as bragging.

The post Impostor syndrome talk: FAQs and follow-ups appeared first on

by Major Hayden at September 02, 2015 03:34 PM

September 01, 2015

Everything Sysadmin

FreeBSD Journal Reviews TPOSANA

Greg Lehey wrote an excellent review of The Practice of System and Network Administration in the new issue of The FreeBSD journal. Even though the book isn't FreeBSD-specific, I'm glad FJ was drawn to reviewing the book.

For more about the FreeBSD Journal, including how to subscribe or purchase single issues, visit their website:

I'm a subscribed to the journal and I highly recommend it. The articles are top notch. Even if you don't use FreeBSD, the articles are a great way to learn about advanced technology and keep up with the industry.

September 01, 2015 05:00 PM

Anton Chuvakin - Security Warrior

Monthly Blog Round-Up – August 2015

Here is my next monthly "Security Warrior" blog round-up of top 5 popular posts/topics this month:
  1. Why No Open Source SIEM, EVER?” contains some of my SIEM thinking from 2009. Is it relevant now? Well, you be the judge.  Current popularity of open source log search tools, BTW, does not break the logic of that post. Succeeding with SIEM requires a lot of work, whether you paid for the software, or not. That – and developing a SIEM is much harder than most people think  [274 pageviews]
  2. Simple Log Review Checklist Released!” is often at the top of this list – the checklist is still a very useful tool for many people. “On Free Log Management Tools” is a companion to the checklist (updated version) [101 pageviews]
  3. My classic PCI DSS Log Review series is always popular! The series of 18 posts cover a comprehensive log review approach (OK for PCI DSS 3.1 as well), useful for building log review processes and procedures , whether regulatory or not. It is also described in more detail in our Log Management book and mentioned in our PCI book (just out in its 4th edition!) [95+ pageviews to the main tag]
  4. “SIEM Resourcing or How Much the Friggin’ Thing Would REALLY Cost Me?” is a quick framework for assessing the SIEM project (well, a program, really) costs at an organization (much more details on this here in this paper). [94 pageviews]
  5. Top 10 Criteria for a SIEM?” came from one of my last projects I did when running my SIEM consulting firm in 2009-2011 (for my recent work on evaluating SIEM, see this document [2015 update]) [74 pageviews out of a total of 4157 views]
In addition, I’d like to draw your attention to a few recent posts from my Gartner blog:
Current research on VA tools and VM practices:
Current maverick research on AI/smart machines risks:
Past research on cloud security monitoring:
Miscellaneous fun posts:

(see all my published Gartner research here)
Also see my past monthly and annual “Top Popular Blog Posts” – 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014.
Disclaimer: most content at SecurityWarrior blog was written before I joined Gartner on Aug 1, 2011 and is solely my personal view at the time of writing. For my current security blogging, go here.

Previous post in this endless series:

by Anton Chuvakin ( at September 01, 2015 04:16 PM

Sean's IT Blog

What’s New in VMware Horizon 6.2–User Experience

One of the areas where Horizon 6.2 has a lot of improvements is in the User Experience category.  The new version adds new features as well as brings a few older features out of tech preview.

Client Drive Redirection for VDI and RDSH

Client Drive redirection for Windows was in Tech Preview in Horizon 6.1.1.  It officially comes out of Tech Preview in Horizon 6.2, and it is now supported on both Windows and Mac clients.  It is also available as a tech preview for Linux clients.

This feature, when installed on the virtual desktop, allows users to remotely access files and data that might have stored on their local PC.  It utilizes compression and encryption when transferring files from the endpoint into the virtual desktop or server. 

Windows 10 Support

Although Windows 10 was officially supported on vSphere 6 on Day 1, it wasn’t supported in Horizon.  Virtual desktops built on Windows 10 would work, but there limits to what you could do, and other components of the Horizon Suite were not designed to work with or support it.

Horizon 6.2 has full support for Windows 10.  The Horizon Agent and Client are supported.  This includes Smart Card authentication support.

Windows 10 is only supported when running ESXi 5.5 Update 3 or ESXi 6.0 Update 1.

File Type Associations for Published Apps

There are times when I may want to allow a user to launch an application or work with files without installing the required applications on their machines.  In these cases, the user would then have to log into Horizon, launch the application, and then navigate to the network location where the file was stored.

But what if I could register a file handler in Windows that would allow me to double click on that file and have it launch the remote application automatically?  Horizon 6.2 now adds this capability.

In order to improve the user experience when opening files remotely, a data compression algorithm is utilized when transferring the files up to the remote host.  This transfer is also protected with SHA 256 encryption for when clients are remotely accessing the remote application over the Internet.

Mac OSX and IOS Support

Horizon Client 3.5 will be supported on OSX 10.11 and IOS 9.

Biometric Authentication

The Horizon Client for IOS will support biometric authentication.  This feature will allow users to store their credentials in Keychain and utilize their fingerprints to sign into their virtual desktops or published applications.  Administrators can also define polices for who can use this feature from with the Horizon Administrator console.

This feature is only supported with Horizon 6.2 when using Horizon Client 3.5.  The mobile device must also be running IOS 8 or IOS 9.

by seanpmassey at September 01, 2015 01:04 PM


OpenSSL Security: A Year in Review

Over the last 10 years, OpenSSL has published advisories on over 100 vulnerabilities. Many more were likely silently fixed in the early days, but in the past year our goal has been to establish a clear public record.

In September 2014, the team adopted a security policy that defines how we handle vulnerability reports. One year later, I’m very happy to conclude that our policy is enforced, and working well.

Our policy divides vulnerabilities into three categories, and defines actions for each category: we use the severity ranking to balance the need to get the fix out fast with the burden release upgrades put on our consumers.

  • HIGH severity issues affect common configurations and are likely to be exploitable. The most precious OpenSSL component is the TLS server, and of the four HIGH severity bugs we had in the last year, two were server DoS. The third was the RSA EXPORT downgrade attack, and the fourth a certificate forgery attack, which luckily was discovered and reported to us very fast and so only affected OpenSSL for one release cycle. When a HIGH severity report comes in, we drop whatever we were doing, investigate, develop patches and start preparing for a release. We aim to get the fix out in under a month.
  • MODERATE severity issues are likely to affect some subset of the OpenSSL users in a notable way. Examples from past year include DoS problems affecting clients and servers with client auth, crashes in PKCS7 parsing, and an array of bugs in DTLS. MODERATE issues don’t kick off an immediate release; rather, we pool them together. But we also don’t wait for a HIGH issue to come along (of course we hope one never does). We’ve been releasing pretty regularly on a bi-monthly basis to get the patches out.
  • LOW severity issues include crashes in less common parts of the API and problems with the command-line tool (which you shouldn’t be using for security purposes!). For those, we’re reasonably confirmed that usage patterns that could lead to exploitation are rare in practice. We do due CVE diligence on every issue that may have a security impact, but in order to reduce the complexity of release and patch management, we commit these fixes immediately to the public git repository.

The graph below (raw data) shows the number of days from first report until the security release for each of the CVEs of the past year. You can see the policy in action: serious vulnerabilities do get fixed faster. (The first 9 issues were released in August 2014, just before adopting the new policy, and don’t have a severity ranking.)

The acceptable timeline for disclosure is a hot topic in the community: we meet CERT’s 45-day disclosure deadline more often than not, and we’ve never blown Project Zero’s 90-day baseline. Most importantly, we met the goal we set ourselves and released fixes for all HIGH severity issues in well under a month. We also landed mitigation for two high-profile protocol bugs, POODLE and Logjam. Those disclosure deadlines weren’t under our control but our response was prepared by the day the reports went public.

We’ve also made mistakes. Notably, the RSA EXPORT man-in-the-middle attack didn’t get the attention or execution speed it deserved. We underestimated the impact and gave it the LOW treatment, only reclassifying it to a HIGH in a later advisory, once we realised how prevalent EXPORT cipher suite support still was. A couple of times, we scrambled to get the release out and introduced new bugs in the process: better release testing is definitely something we need to work on, and we’re grateful to everyone who’s helped us with continuous integration tests.

Of course, the true goal is to not have any CVEs in the first place. So I can’t say it’s been a good year: too many bugs are still being found in old code. But we’re working hard to improve the code quality going forward, and we’ve set the baseline.

Finally, a special thanks to all the security researchers who’ve sent reports to the quality of reports is generally very high and your collaboration in analysing the vulnerabilities has been tremendously helpful.

September 01, 2015 12:47 PM

Ansible-cmdb v1.4: a host overview generator for ansible-managed hosts

Ansible-cmdb takes the output of Ansible's setup module and converts it into a static HTML overview page containing system configuration information. It supports multiple templates and extending information gathered by Ansible with custom data.

You can visit the Github repo, or view an example output here.

This is the v1.4 release of ansible-cmdb, which brings a bunch of bug fixes and some new features:

  • Support for host inventory patterns (e.g. foo[01:04]
  • Support for 'vars' and 'children' groups.
  • Support passing a directory to the -i param, in which case all the files in that directory are interpreted as one big hosts file.
  • Support for the use of local jquery files instead of via a CDN. Allows you to view the hosts overview in your browser using file://. See for info on how to enable it (hint: ansible-cmdb -p local_js=1).
  • Add -f/–fact-caching flag for compatibility with fact_caching=jsonfile fact dirs (Rowin Andruscavage).
  • The search box in the html_fancy template is now automatically focussed.
  • Show memory to one decimal to avoid "0g" in low-mem hosts.
  • Templates can now receive parameters via the -p option.
  • Strip ports from hostnames scanned from the host inventory file.
  • Various fixes in the documentation.
  • Fixes for Solaris output (memory and disk).

I would like to extend my gratitude to the following contributors:

  • Sebastian Gumprich
  • Rowin Andruscavage
  • Cory Wagner
  • Jeff Palmer
  • Sven Schliesing

If you've got any questions, bug reports or whatever, be sure to open a new issue on Github!

by admin at September 01, 2015 07:07 AM

August 29, 2015

Feeding the Cloud

Letting someone ssh into your laptop using Pagekite

In order to investigate a bug I was running into, I recently had to give my colleague ssh access to my laptop behind a firewall. The easiest way I found to do this was to create an account for him on my laptop and setup a pagekite frontend on my Linode server and a pagekite backend on my laptop.

Frontend setup

Setting up my Linode server in order to make the ssh service accessible and proxy the traffic to my laptop was fairly straightforward.

First, I had to install the pagekite package (already in Debian and Ubuntu) and open up a port on my firewall by adding the following to both /etc/network/iptables.up.rules and /etc/network/ip6tables.up.rules:

-A INPUT -p tcp --dport 10022 -j ACCEPT

Then I created a new CNAME for my server in DNS:   3600    IN  CNAME

With that in place, I started the pagekite frontend using this command:

pagekite --clean --isfrontend --rawports=virtual --ports=10022

Backend setup

After installing the pagekite and openssh-server packages on my laptop and creating a new user account:

adduser roc

I used this command to connect my laptop to the pagekite frontend:

pagekite --clean --service_on=raw/

Client setup

Finally, my colleague needed to add the folowing entry to ~/.ssh/config:

  CheckHostIP no
  ProxyCommand /bin/nc -X connect -x %h:10022 %h %p

and install the netcat-openbsd package since other versions of netcat don't work.

On Fedora, we used netcat-openbsd-1.89 successfully, but this newer package may also work.

He was then able to ssh into my laptop via ssh

Making settings permanent

I was quite happy settings things up temporarily on the command-line, but it's also possible to persist these settings and to make both the pagekite frontend and backend start up automatically at boot. See the documentation for how to do this on Debian and Fedora.

August 29, 2015 09:20 PM

August 28, 2015

Evaggelos Balaskas


This is a list with podcasts I listen on a regular base

Tag(s): podcast

August 28, 2015 09:54 PM

Racker Hacker

Build a high performance KVM hypervisor on Rackspace’s OnMetal servers

I received some good feedback about my post on systemd-networkd and bonded interfaces on Rackspace’s OnMetal servers, and I decided to write about another use case. Recent product updates allow you to attach a Cloud Block Storage volume, and this opens up quite a few new possibilities for deployments.

So why not create a high-performance KVM hypervisor on an OnMetal server? Let’s do this.


WHOA THERE. These are amazing servers and because of that, they’re priced much differently than Cloud Servers are. Be sure to review the pricing for OnMetal and Cloud Block Storage before going through this guide. Don’t end up with an unexpected bill by building one of these servers and forgetting to destroy it.

Building the server

We can build our server using command line tools. One of my tools, supernova, makes this quite easy. My IAD environment is called prodiad and I can boot an OnMetal server like this:

supernova prodiad boot \
  --flavor onmetal-memory1 \
  --image 4c361a4a-51b4-4e29-8a35-3b0e25e49ee1 \
  --key_name personal_servers \
  --poll \

In the command above, I’ve built an OnMetal Memory server. I’ll end up with some hardware like this:

  • Dual Intel Xeon E5-2630 v2 2.6Ghz
  • 12 cores total
  • 512GB RAM
  • 10Gbps connectivity
  • 32GB disk

Everything looks amazing except for the storage — but we’ll fix that soon. I’ve also built the server with Fedora 22 and provided my public ssh key.

Wait a few minutes after running the supernova command and you should be back to a prompt. Verify that your new OnMetal server is pinging, but keep in mind it may still be in the process of booting up or configuring itself.

Adding storage

Getting additional storage for an OnMetal server is done in two steps: provisioning the LUN and attaching it to the host. This is a bit easier in Cloud Servers since the actual attachment is done behind the scenes. You end up with a disk that attaches itself to the virtual machine at the hypervisor layer. OnMetal is a little different, but the process is still very straightforward.

Let’s start by making four 100GB SSD volumes. We will eventually put these into a RAID 10 volume.

for i in `seq 1 4`; do
    supernova prodiad volume-create \
      --display-name onmetal-kvm-${i} \
      --volume-type SSD \

We can list our new volumes:

$ supernova prodiad volume-list
| ID                                   | Status    | Display Name  | Size | Volume Type | Attached to |
| 0beb1f81-eb04-4aca-9b14-c952f9eb81e2 | available | onmetal-kvm-4 | 100  | SSD         |             |
| 83b9d6d9-e7eb-4b53-9342-fa2fd3670bb4 | available | onmetal-kvm-3 | 100  | SSD         |             |
| a593cbbe-089f-4ede-81f4-003717b2309f | available | onmetal-kvm-2 | 100  | SSD         |             |
| 2c51e09f-d984-4de5-8852-c0f9c6176e00 | available | onmetal-kvm-1 | 100  | SSD         |             |

It’s now time to attach our volumes to our OnMetal server. Let’s get our OnMetal server’s UUID:

$ supernova prodiad list --name kvm-onmetal --minimal
[SUPERNOVA] Running nova against prodiad... 
| ID                                   | Name        |
| 6a80d0b9-ce3e-4693-bedb-d843fea7cb0b | kvm-onmetal |

Now we’re ready to attach the volumes:

supernova prodiad volume-attach $ONMETAL_UUID 2c51e09f-d984-4de5-8852-c0f9c6176e00
supernova prodiad volume-attach $ONMETAL_UUID a593cbbe-089f-4ede-81f4-003717b2309f
supernova prodiad volume-attach $ONMETAL_UUID 83b9d6d9-e7eb-4b53-9342-fa2fd3670bb4
supernova prodiad volume-attach $ONMETAL_UUID 0beb1f81-eb04-4aca-9b14-c952f9eb81e2

Let’s log into the OnMetal server and get it ready. Install the iscsi-initator-utils package and set up the services:

dnf -y install iscsi-initiator-utils
systemctl enable iscsid
systemctl start iscsid

Our iSCSI IQN data is in our OnMetal server’s metadata. Grab your metadata JSON with this command:

supernova prodiad show 6a80d0b9-ce3e-4693-bedb-d843fea7cb0b | grep metadata

If you copy/paste the JSON data into a file, you can use Python to make the JSON easier to read:

cat iscsi_metadata.json | python -m json.tool

Start by putting your server’s initiator name into a file. It should be called `initiator_name` in the JSON data.

echo > /etc/iscsi/initiatorname.iscsi

Do the iSCSI logins for each `target_iqn` and `target_portal` in your JSON output. It should look something like this each time:

# iscsiadm -m discovery --type sendtargets --portal $TARGET_PORTAL
# iscsiadm -m node --targetname=$TARGET_IQN --portal $TARGET_PORTAL --login

When you’re all done, you should have four new disks:

# ls /dev/disk/by-path/

## Building the RAID volume
We can build the raid volume using the paths from above to prevent against device name changes later. Let’s make a RAID 10 volume:

dnf -y install mdadm
mdadm --create /dev/md0 --level=10 --raid-devices=4 /dev/disk/by-path/*

Check the status of the new RAID volume:

# cat /proc/mdstat 
Personalities : [raid10] 
md0 : active raid10 sdd[3] sdc[2] sdb[1] sde[0]
      209584128 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
      [>....................]  resync =  0.7% (1534400/209584128) finish=15.8min speed=219200K/sec

Come on, our storage volumes are faster than that. Let’s speed it up a bit:

# sysctl -w
# cat /proc/mdstat 
Personalities : [raid10] 
md0 : active raid10 sdd[3] sdc[2] sdb[1] sde[0]
      209584128 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
      [====>................]  resync = 21.1% (44229312/209584128) finish=2.9min speed=925564K/sec

That’s more like it. Let’s put a XFS filesystem on the volume and get it mounted:

dnf -y install xfsprogs
mkfs.xfs /dev/md0
mkdir /mnt/raid
echo "/dev/md0 /mnt/raid xfs defaults,noatime 0 1" >> /etc/fstab
mount -a

## Getting KVM going
It’s time to get packages updated and installed:

dnf -y upgrade
dnf -y install libvirt libvirt-daemon* virt-install virt-manager xorg-x11-xauth gnome-icon-theme gnome-themes-standard dejavu*
systemctl start libvirtd
systemctl enable libvirtd

We can create a qcow volume and begin installing Fedora into a virtual machine:

qemu-img create -f qcow2 /mnt/raid/fedora-kvm.qcow2 20G
virt-install --name=fedora-kvm --ram=16384 \
    --vcpus=4 --os-variant=fedora21 --accelerate \
    --hvm --network network=default \
    --disk /mnt/raid/fedora-kvm.qcow2 \
    --location \
    --noautoconsole --graphics vnc --autostart

Logout and then ssh to the server again, this time with `-Y` for X forwarding. Run `virt-manager` and verify that the VM is running.


Double-click on the virtual machine listed there and the anaconda installer should be on the screen.

OnMetal KVM VM

Let the installation complete and you’ll have a KVM virtual machine ready to roll!

Additional thoughts

Obviously, this is a very manual process. It could be automated with scripts, or an orchestration framework, like Ansible. In addition, deployment of virtual machines could be automated with OpenStack. However, my goal here was to demonstrate a new use case for OnMetal servers. I’ll add the automation to my long list of to-do’s.

The post Build a high performance KVM hypervisor on Rackspace’s OnMetal servers appeared first on

by Major Hayden at August 28, 2015 02:00 PM

August 27, 2015

Carl Chenet

Liens intéressants Journal du hacker semaine #35

Suivez-moi aussi sur Diaspora*diaspora-banner ou Twitter  ou sur


Pour cette 35ème semaine de 2015, 5 liens intéressants que vous avez peut-être ratés, relayés cette semaine par le Journal Du Hacker, votre source d’informations pour le Logiciel Libre francophone !



Pour ne plus rater aucun article de la communauté francophone, voici :

De plus le site web du Journal du hacker est « adaptatif (responsive) ». N’hésitez pas à le consulter depuis votre smartphone ou votre tablette !

Le Journal du hacker fonctionne de manière collaborative, grâce à la participation de ses membres. Rejoignez-nous pour proposer vos contenus à partager avec la communauté du Logiciel Libre francophone et faire connaître vos projets.

Et vous ? Qu’avez-vous pensé de ces articles ? N’hésitez pas à réagir directement dans les commentaires de l’article sur le Journal du hacker ou bien dans les commentaires de ce billet :)



by Carl Chenet at August 27, 2015 08:30 PM

Racker Hacker

Fedora 23 Alpha in

Fedora 23’s Alpha release was announced earlier this month and work is underway for the beta release. The full list of dates for the Fedora 23 release is in the Fedora wiki.

If you’d like to try Fedora 23 Alpha a little sooner, check out I added support for Fedora 23 in the menus last night.

Quick start

If you want to get underway quickly, simply download the ISO and attach it to a virtual machine:


When it boots, you’ll be able to select Fedora 23’s Alpha release from the menus. The Workstation, Atomic, and Server images are available.

Fedora 23 alpha


The post Fedora 23 Alpha in appeared first on

by Major Hayden at August 27, 2015 01:03 PM

August 26, 2015

Carl Chenet

Retweet 0.2 : bump to Python 3

Follow me on  or Twitter  or Diaspora*diaspora-banner

Don’t know Retweet? My last post about it introduced this small Twitter bot whichs just retweets (for now) every tweets from a Twitter account to another one.


Retweet was created in order to improve the Journal du hacker Twitter account. The Journal du hacker is a Hacker News-like French-speaking website.


Especially useful to broadcast news through a network of Twitter accounts, Retweet was improved to bump Python version to 3.4 and to improve pep8 compliance (work in progress).

The project is also well documented and should be quite simple to install, configure and use.

After my first blog post about Retweet, new users gave me feedback about it and I now have great ideas for future features for the next release.


What about you? If you try it, please tell me what you think about it, opening a bug request or ask for new features. Or just write your comment here ;)

by Carl Chenet at August 26, 2015 09:01 PM

Standalone Sysadmin

Great Open Positions at Northeastern CCIS

I’ve landed in Los Angeles, and I’m getting settled in temporary housing until I find my own place, but it’s been a really busy couple of weeks, and I just realized that I didn’t get a chance to post about the open positions that my (now former) team has.

First, more obviously, there’s my old position, that of the Networking & Virtualization Administrator. The position is officially posted on Northeastern’s Careers page, but I can tell you that you’d be responsible for a medium-sized relatively flat network infrastructure. There are a few dozen VLANs, all statically routed from the core switches, and around a thousand lit switchports. The hardware is mostly Cisco Catalyst, with the core being Cisco Nexus 5548s, although there are some virtual PFsense boxes running around too.  You would be working with the (pretty friendly and competent) central ITS network admin to coordinate staff and faculty moves around the infrastructure, and with the university’s security officer (who is also surprisingly friendly, given his line of work) whenever something weird pops up.

The role is also responsible for the VMware cluster, which currently consists of around 15 ESXi nodes and two vCenter instances (one for “production” use which has the vSphere Essentials Plus license) and the educational cluster, built out using VMware Academic licenses for classroom and academic use. They’re backed by NetApp and Nimble storage, and it’s this part of the job responsibilities that gives you a little more creativity to solve problems, since professors usually want interesting things. I’ve built some useful stuff in PowerShell, but there’s no reason you have to use that long-term, if you want to solve the problems yourself.

Anyway, I really enjoyed my time in this position, and to be honest, I really miss the other staff members and students there.

In addition, the CCIS staff is growing. We got a new dean a little over a year ago, and one of the things she wants to do is to offer management of researchers’ clusters in a more active manner, so we are looking for another Linux sysadmin (pretty much all of the researchers do work on Linux).

This position will involve a lot working with our current Linux admin to bring over the technology he has built to deal with our “managed” machines to help with our “unmanaged” or “soon to be managed” researcher-owned machines. Basically, there’s nothing like this right now, so you would be inventing the role as you go. Exciting! Challenging! Rewarding!

Anyway, please, if you’re looking for a position in Boston somewhere, take a look at Northeastern. It’s easy to get to, there’s free tuition for you, your spouse, and your children, and I feel like the staff that I worked with there are my family, and I miss them :-)

If you have any questions, please drop me an email and I’ll be happy to help. Thanks!

by Matt Simmons at August 26, 2015 03:11 PM

Anton Chuvakin - Security Warrior

August 23, 2015

Evaggelos Balaskas

forwarding logs with Fluentd

Server_A —> Server_B —> Server_C

Let’s say that we have our elasticsearch/kibana setup on Server_C
but Server_A can’t talk to Server_C.


# tail /etc/rsyslog.d/20_central_logging.conf 

*.*      @
& ~


install fluentd

# wget -c
# rpm -ivh td-agent-2.2.1-0.el6.x86_64.rpm

configure fluentd

# vim /etc/td-agent/td-agent.conf
  type syslog
  port 42185
  tag  rsyslog

<match ***>
  type forward
  send_timeout 10s
  recover_wait 10s
  heartbeat_interval 1s
  phi_threshold 16
  hard_timeout 60s


Server C

install fluentd

# wget -c
# rpm -ivh td-agent-2.2.1-0.el6.x86_64.rpm

configure fluentd

# vim /etc/td-agent/td-agent.conf
<match ***>
  type elasticsearch
  flush_interval 10s # for testing
  logstash_format true

PLZ Dont forget your iptables rules !!!!

Tag(s): Fluentd

August 23, 2015 06:06 PM

August 21, 2015

Colin Percival

Tarsnap $1000 exploit bounty

For somewhat over four years, Tarsnap has been offering bounties for bugs found in the Tarsnap code. Two thirds of the bounties Tarsnap has paid out have been $1 each for cosmetic bugs (e.g., typos in source code comments), and a quarter of the bugs have been $10 each for harmless bugs — mostly memory leaks in error paths where the tarsnap client is about to exit anyway — but there have also been some more serious bugs: Several build-breakage bugs ($20 each); a variety of cases where tarsnap behaviour is wrong in a user-visible — but generally very obscure — way ($50 each); a few crashes ($100); and of course the critical crypto bug which first convinced me to offer bounties.

Most bugs are straightforward, but occasionally one comes up which is not so clear in its impact. Such is the case with a bug which is fixed in tarsnap 1.0.36. This bug causes the NUL string termination byte to overflow the heap-allocated buffer used for paths of objects examined as tarsnap traverses a directory tree; such one-byte heap overflows have been shown to be exploitable in the past. In the case of tarsnap, I will be very surprised if it turns out to be possible to cause anything worse than a crash, but I can't absolutely rule out the possibility.

In light of this, Tarsnap is offering a $1000 exploit bounty: The first person before the end of 2015 who can convincingly demonstrate a serious exploitation of this bug will receive $1000. While there are many organizations which pay more than this for exploits, I think this is a reasonable prize: After all, I'm already telling you what the bug is which you need to exploit! Fine print: No bounty if you're in Iran, North Korea, or some other problem countries. Bounties are awarded at my sole discretion; in particular, I get to decide whether the "convincingly demonstrate" and "serious exploitation" conditions are satisfied. Payment by US dollar check or paypal. To avoid races, contact me before publishing anything. If you can't accept cash prizes, the bounty can be donated to a mutually-acceptable charity of your choice.

August 21, 2015 02:00 PM

CFEngine fixin my FreeNAS

I recently built a new file server and I based it on the well renowned FreeNAS by iXsystems. It’s been pretty solid over the past few weeks but today I ran into an issue. The web ui stopped responding. Actually it turned out that the django service had stopped. Well, that was the perfect opportunity to use CFEngine to make sure I never have the issue again.


I grabbed the cfengine community 3.7.0 package for Freebsd 9.3 package from (thanks guys!) and it installed without issue.

pkg_add cfengine-community-3.7.0_1-freebsd_9.x_amd64.tbz

I just wanted to experiment locally instead of bootstrapping to a policy server so I grabbed the masterfiles source tarball for 3.7.0 and installed the masterfiles policy framework.

tar zxvf cfengine-masterfiles-3.7.0-2.tar.gz
cd cfengine-masterfiles-3.7.0/
make install

Since I am only going to have local policy for now I went ahead and linked inputs to masterfiles.

rm -rf /var/cfengine/inputs
ln -s /var/cfengine/masterfiles /var/cfengine/inputs

And then I enabled cfengine.

cfengine3_enable="YES" >> /etc/rc.conf
service start

I enabled autorun for convenience.

sed -i 's/.*services_autorun.*expression.*/      "services_autorun" expression => "any";/' /var/cfengine/masterfiles/controls/3.7/

And then I installed this policy into services/autorun.

wget -O /var/cfengine/masterfiles/services/autorun/

Now any time django decides to die, CFEngine will come along and fix it up.

[root@freenas] ~# service django stop
Stopping django.
Waiting for PIDS: 17087.
[root@freenas] ~# cf-agent -K
[root@freenas] ~# service django status
django is running as pid 17203.
[root@freenas] ~# tail /var/log

Maybe I’ll do some Software Defined Storage :-p

by Nick Anderson at August 21, 2015 04:51 AM

August 20, 2015

League of Professional System Administrators

LISA 2015 Discount Code

Summer is winding down and we are moving into Fall which means it is time to make your plans to attend the premier system administration conference.  Yes the LISA'15 website is live at and as a member of LOPSA you get a $45 discount to attend.

read more

by lopsawebstaff at August 20, 2015 10:24 PM

Carl Chenet

Liens intéressants Journal du hacker semaine #34

Suivez-moi aussi sur Diaspora*diaspora-banner ou Twitter  ou sur


Pour cette 34ème semaine de 2015, 5 liens intéressants que vous avez peut-être ratés, relayés cette semaine par le Journal Du Hacker, votre source d’informations pour le Logiciel Libre francophone !



Proposed Debian Logo

Pour ne plus rater aucun article de la communauté francophone, voici :

De plus le site web du Journal du hacker est « adaptatif (responsive) ». N’hésitez pas à le consulter depuis votre smartphone ou votre tablette !

Le Journal du hacker fonctionne de manière collaborative, grâce à la participation de ses membres. Rejoignez-nous pour proposer vos contenus à partager avec la communauté du Logiciel Libre francophone et faire connaître vos projets.

Et vous ? Qu’avez-vous pensé de ces articles ? N’hésitez pas à réagir directement dans les commentaires de l’article sur le Journal du hacker ou bien dans les commentaires de ce billet :)

by Carl Chenet at August 20, 2015 08:30 PM

August 19, 2015

Rudimentary compliance report for CFEngine

In CFEngine community you don’t have a web GUI with compliance report. You can get them via EvolveThinking’s Delta Reporting, but if you can’t for any reason, you need to find another way.

A poor man’s compliance report at the bundle level can be extracted via the verbose output. This is how I’ve used it to ensure that a clean-up change in the policies didn’t alter the overall behavior:

cf-agent -Kv 2>&1 | perl -lne 'm{verbose: (/.+): Aggregate compliance .+ = (\d+\.\d%)} && print "$1 ($2)"'

These are the first ten lines of output on my workstation:

bronto@brabham:~$ sudo cf-agent -Kv 2>&1 | perl -lne 'm{verbose: (/.+): Aggregate compliance .+ = (\d+\.\d%)} && print "$1 ($2)"' | head -n 10
/default/banner (100.0%)
/default/inventory_control (100.0%)
/default/inventory_autorun/methods/'proc'/default/cfe_autorun_inventory_proc (100.0%)
/default/inventory_autorun/methods/'fstab'/default/cfe_autorun_inventory_fstab (100.0%)
/default/inventory_autorun/methods/'mtab'/default/cfe_autorun_inventory_mtab (100.0%)
/default/inventory_autorun/methods/'dmidecode'/default/cfe_autorun_inventory_dmidecode (100.0%)
/default/inventory_autorun (100.0%)
/default/inventory_linux (100.0%)
/default/inventory_lsb (100.0%)
/default/services_autorun (100.0%)

Not much, but better than nothing and a starting point anyway. There is much more information in the verbose log that you can extract with something slightly more elaborated than this one-liner. Happy data mining, enjoy!

Tagged: cfengine, Configuration management, DevOps, one liners, Perl, Sysadmin

by bronto at August 19, 2015 01:28 PM

LZone - Sysadmin

Debugging hiera-eyaml Encryption, Decryption failed

When Hiera works without any problems everything is fine. But when not it is quite hard to debug why it is not working. Here is a troubleshooting list for Hiera when used with hiera-eyaml-gpg.

hiera-eyaml-gpg Decryption failed

First check your GPG key list
gpg --list-keys --homedir=<.gnupg dir>
Check that at least one of the keys listed is in the recipients you use for decrypting. The recipients you used are either listed in your Hiera/Eyaml config file or in a file referenced from there.

To verify what you active config is run eyaml in tracing mode. Note that the "-t" option is only available in newer Eyaml versions (e.g. 2.0.5):
eyaml decrypt -v -t -f somefile.yaml
Trace output
[hiera-eyaml-core]           (Symbol) trace_given        =        (TrueClass) true              
[hiera-eyaml-core]           (Symbol) gpg_always_trust   =       (FalseClass) false             
[hiera-eyaml-core]           (Symbol) trace              =        (TrueClass) true              
[hiera-eyaml-core]           (Symbol) encrypt_method     =           (String) pkcs7             
[hiera-eyaml-core]           (Symbol) gpg_gnupghome      =           (String) /etc/hiera/.gnupg      
[hiera-eyaml-core]           (Symbol) pkcs7_private_key  =           (String) ./keys/private_key.pkcs7.pem
[hiera-eyaml-core]           (Symbol) version            =       (FalseClass) false             
[hiera-eyaml-core]           (Symbol) gpg_gnupghome_given =        (TrueClass) true              
[hiera-eyaml-core]           (Symbol) help               =       (FalseClass) false             
[hiera-eyaml-core]           (Symbol) quiet              =       (FalseClass) false             
[hiera-eyaml-core]           (Symbol) gpg_recipients_file =           (String) ./gpg_recipients  
[hiera-eyaml-core]           (Symbol) string             =         (NilClass)                   
[hiera-eyaml-core]           (Symbol) file_given         =        (TrueClass) true   
Alternatively try manually enforcing recipients and .gnupg location to make it work.
eyaml decrypt -v -t -f somefile.yaml --gpg-recipients-file=<recipients> --gpg-gnupghome=<.gnupg dir>
If it works manually you might want to add the keys ":gpg-recipients-file:" to hiera.yaml and ensure that the mandatory key ":gpg-gnupghome:" is correct.

Checking Necessary Gems

hiera-eyaml-gpg can be run with different GPG-libraries depending on the version you run. Check dependencies on Github.

A possible stack is the following
gem list
gpgme (2.0.5)
hiera (1.3.2)
hiera-eyaml (2.0.1)
hiera-eyaml-gpg (0.4)
The GEM gpgme additionally needs the C library
dpkg -l "*gpg*"
||/ Name                Version             Beschreibung
ii  libgpgme11          1.2.0-1.2+deb6u1    GPGME - GnuPG Made Easy

Using Correct Ruby Version

Another pitfall is running multiple Ruby versions. Ensure to install the GEMs into the correct one. One Debian consider using "ruby-switch" or manually running "update-alternatives" for "gem" and "ruby".

Ruby Switch

apt-get install ruby-switch
ruby-switch --set ruby1.9.1


# Print available versions
update-alternatives --list ruby
update-alternatives --list gem

# Show current config update-alternatives --display ruby update-alternatives --display gem

# If necessary change it update-alternatives --set ruby /usr/bin/ruby1.9.1 update-alternatives --set gem /usr/bin/gem1.9.1

August 19, 2015 10:47 AM

Debugging dovecot ACL Shared Mailboxes Not Showing in Thunderbird

When you can't get ACL shared mailboxes visible with Dovecot and Thunderbird here are some debugging tipps:
  1. Thunderbird fetches the ACLs on startup (and maybe at some other interval). So for testing restart Thunderbird on each change you make.
  2. Ensure the shared mailboxes index can be written. You probably have it configured like
    plugin {
      acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db
    Check if such a file was created and is populated with new entries when you add ACLs from the mail client. As long as entries do not appear here, nothing can work.
  3. Enable debugging in the dovecot log or use the "debug" flag and check the ACLs for the user who should see a shared mailbox like this:
    doveadm acl debug -u shared/users/box
    • Watch out for missing directories
    • Watch out for permission issues
    • Watch out for strangely created paths this could hint a misconfigured namespace prefix

August 19, 2015 10:20 AM

August 17, 2015

Sarah Allen

designing for fun @altconf 2015

Making your app fun to use requires more than sprinkling a little gamification on top. It requires thoughtful imagination and experimentation. I recently spoke at AltConf: “Designing for Fun” (now on video) highlights some expert perspectives on theories of play and behavioral psychology, and how we can apply these ideas in mobile app design. I also shared some prototyping and customer development techniques, plus how to validate whether a design will actually be fun.

Below is an overview, plus notes and references from the talk. You can also check out the slides.


Most of this talk is about how to design fun experiences, but the title has a double meaning — I talk about an app I’m working on “for fun” (not for “work”). Most of the code was written by my friend and iOS development mentor, John Fox, plus we have a large extended team of people who make it happen. The thing about doing an app outside of your day job means that we work hard to make sure we’re all aligned and are motivated and having fun, since when it stops being fun, we’ll stop playing.

For the first half of this talk, I focus on theories of play and game design, with a couple of examples from my prior work. The second half shows a practical use case from the Mightyverse app we are building now.

For context: Mightyverse is a global community of people sharing language and culture. At it’s heart, there’s a collection of short phrase videos of people who have recorded a phrase in their native language, that is cross-translated into other languages. We have collected tens of thousands of short phrase videos for learning language and we’re building a mobile app to crowdsource recordings from native speakers, while also allowing people to learn new languages.

Notes and References

Sebastian Deterding: Meaning, Autonomy & Mastery. From Google Tech Talk, Getting Gamification Right

Research has identified the chemical dopamine affects learning and memory. Doing something rewarding increases dopamine. Eric Marr at TEDxCCS: Dopamine’s effects on learning and memory

The EPIC Winis an extraordinary outcome that you didn’t believe was even possible until you achieved it — almost beyond your threshold of imagination, something that teaches you what you’re truly capable of. “Gamers always believe that an epic win is possible, and that it’s always worth trying, and trying now.” Jane McGonigal TED talk: Gaming can make a better world

Almost 50% of the world’s languages are at risk. In my talk I said “the people who decide which languages we keep are three years old.” The source for this was linguist David Harrison’s talk Living Languages Digital Dialog. He actually refers to 5, 6, and 7-year olds, who he calls the “true decision makers in communities about whether to keep or abandon a language.” When we as adults show we value a language, the kids learn it.

There are over 6000 languages in the world — this WSJ article is a good reference. The majority of them are spoken by a tiny fraction of the population, and almost 80% of us, speak only 83 languages — I created the visualization for a 2009 blog post: who cares if languages become extinct?

Play needs to be voluntary for it to be fun. I can’t find the reference for this, despite looking many times. Maybe it was in one of these videos or some article I read. If anyone has this reference I would love to read it again!

Frank Smith, a leading authority on linguistics and cognitive psychology, reports that: “Learning is the brain’s primary function, its constant concern, and we become restless and frustrated if there is no learning to be done. We are all capable of huge and unsuspected learning accomplishments without effort.” (Insult to Intelligence: The Bureaucratic Invasion of Our Classrooms).

“Fun is just another word for learning under optimal conditions.” — Raph Koster

I believe that software design is teaching. We want to make it so people can effortlessly learn how to use a piece of software. Every little bit of learning should give people powers they can use repeatedly toward achieving their own goals.

This is the first dialog box I ever designed was for PACo Producer. I scanned this from the documentation:
Mac Plus era dialog box with a lot of text and some dotted underlined numbers and filenames.
The little dotted link is a pre-web hyperlink. Easy to learn and remember since it is related visually to a real-world paper form. It is worth teaching someone something to give them a powerful new tool. It seems like this was successful since we used the same pattern in After Effects, and it has persisted over 20 years later, likely having survived many usability tests.

I wondered… After Effects has a lot of complex UI, which is pretty overwhelming at first glance. I asked a colleague of mine who has been a user of After Effects since 1.0: is it fun to use?

“I love it. Anything is possible when i use it…I can dream up something and then make it real.
It keeps improving and with each improvement i’m able to communicate my ideas a little faster, a little more clearly…” — Paul Lundahl

After Effect screen shot was composite from:

I shared an example from the development of Flash video, circa 2001. The overwhelming assumption at that time for mutliplayer games or web video conferencing was that the experience would start with a login screen. It was Jonathan Gay who really challenged that assumption by insisting that we make it so Flash applications could enable real-time human-to-human interactions without requiring a name or password.

People already know how to interact with each other. Don’t make people make decisions until they have to, or you risk that they will make the decision to leave your app!

What are these optimal conditions for learning?

Stress actually inhibits learning. The optimal state of mind for learning is “relaxed alertness” Geoffrey Caine & Renate N. Caine Making Connections: Teaching and the Human Brain

In early 1900s, Lev Vygotsky studied imaginative play in children and observed that children will subordinate their own wants to the greater pleasure of following the rules. “The essential attribute of play is a rule that has become a desire.” (Vygotsky, Mind in Society)

Stephanie Morgan Creative Mornings talk “Gamification Sucks” Computer games stimulate the brain’s reward system to produce dopamine — in addition to making us feel good, this chemical seems to be the physical basis for learning. Research has shown that the introduction of chance into any reward system increases dopamine production.

Play Testing

All of us are not in the target audience for our app. We need to be careful about interpreting our own responses to our inventions, but often we have some characteristics of our own target audience and can be the very first play testers. In our Mightyverse team, Iku is always seeking to improve her english, and Paul and I have taken some Japanese classes, so we figured we were good for a first test. Our goal to is get people to have fun actually speaking the language they are learning. We intentionally designed without a point system, since we believe that language learning can be intrinsically fun. We tested this theory by making a game without an external point system, just tracking whether the players learned the phrases together.

“Shut up and sit in the corner and watch.
See if people who play your game are having fun
and playing the way you expect,
and are able to learn the rules easily.”
Cooperation and Engagement: What can board games teach us? Google Tech Talk by Matt Leacock

Be careful who you pick to play test your game — not just your brother, your wife or husband, your kid, unless they are in your target audience.

The first Mightyverse game play test with real audience was at SF Babel: 3 decks, written on index cards with 200 phrases in English, Spanish and Japanese.

Games are more fun when people are expecting to play a game. The game state, with its suspension of disbelief, and the rules create this alternate reality within which you can have fun. Games are more often played in the living room or around a kitchen table. Play test with your friends. You want your first play tests to be with people who will still play your game again, even if the first experience is frustrating or boring. Your friends will always play a game with you. Of course, they need to be part of your target audience. If you don’t have any friends who are part of your target audience, go out and meet people in your target audience and make friends with them.

Another great way to find people in your target audience is a crowd-funding campaign. If people will pay for something that doesn’t yet exist, then they probably want it. We knew we needed to do a lot more play testing, so we decided to commit to printing the game in order to find more people in our target audience with the campaign, setting us firmly on the path of learning about our future customers and validating our theories about how to make language learning fun. We made this video during the campaign — it’s not just marketing, it’s learning about what resonates with people.

Shigeru Miyamoto, famed Nintendo game designer who created Super Mario and the Wii, is known for designing for the expression on someone’s face when they play the game — they should smile and be happy, not frustrated. With the Wii, he designs for everyone in the room, not just the game player.

Our goal is to get people to have fun learning the language. The card game succeeds in that at a small scale. Now that we have developed our own model of language learning and have evidence that it is fun, we can scale our efforts by designing a mobile app — in many ways it will be completely different, but we can apply those core principles that we have validates.

We first built a very small app that only did phrase recordings, and we created a collaborative activity where people would record different phrases of the Martin Luther King “I have a dream” speech translated into Spanish. We wanted to learn if we could construct and activity and get both friends and strangers to engage with our app. We found people that actually seemed to have fun recording phrases. We noticed that some people got their friends involved, and we designed around the parts of the experience that seemed most fun and engaging.

It can be emotionally difficult to test your app when it’s not finished, but that is exactly when you need to test it. It is SO important to start engaging people in the experience. One way to look at it is that the play-test itself is a game.

Here are the rules that I use to make it fun:

  • Shut up and Watch
  • Take Notes
  • Take Photos
  • Resist providing answers.
  • Ask questions.

Questions I ask:

  • What do you think this app is for?
  • What did you expect to happen?
  • Did you have fun? What part of it was fun?
  • Did you learn anything?

Designing for Fun Slides

Photo Credits:
RyanMcGuire, Cats Jumping Playfully
Tambako The Jaguar, Playing Cubs
Tambako The Jaguar, Playing with mom II
Juhan Sonin Follow, Udo finds Viggo
Steven Depolo, Children Twister Party

The post designing for fun @altconf 2015 appeared first on the evolving ultrasaurus.

by sarah at August 17, 2015 12:01 PM

August 16, 2015

Cryptography Engineering

The network is hostile

Yesterday the New York Times and ProPublica posted a lengthy investigation based on leaked NSA documents, outlining the extensive surveillance collaboration between AT&T and the U.S. government. This surveillance includes gems such as AT&T's assistance in tapping the main fiber connection supporting the United Nations, and that's only the start.

The usual Internet suspects are arguing about whether this is actually news. The answer is both yes and no, though I assume that the world at large will mostly shrug at this point. After all, we've learned so much about the NSA's operations at this point that we're all suffering from revelation-fatigue. It would take a lot to shock us now.

But this isn't what I want to talk about. Instead, the effect of this story was to inspire me to look back on the NSA leaks overall, to think about what they've taught us. And more importantly -- what they mean for the design of the Internet and our priorities as security engineers. That's what I'm going to ruminate about below.

The network is hostile

Anyone who has taken a network security class knows that the first rule of Internet security is that there is no Internet security. Indeed, this assumption is baked into the design of the Internet and most packet-switched networks -- systems where unknown third parties are responsible for handling and routing your data. There is no way to ensure that your packets will be routed as you want them, and there's absolutely no way to ensure that they won't be looked at.

Indeed, the implications of this were obvious as far back as ARPANET. If you connect from point A to point B, it was well known that your packets would traverse untrusted machines C, D and E in between. In the 1970s the only thing preserving the privacy of your data was a gentleman's agreement not to peek. If that wasn't good enough, the network engineers argued, you had to provide your own security between the endpoints themselves.

My take from the NSA revelations is that even though this point was 'obvious' and well-known, we've always felt it more intellectually than in our hearts. Even knowing the worst was possible, we still chose to believe that direct peering connections and leased lines from reputable providers like AT&T would make us safe. If nothing else, the NSA leaks have convincingly refuted this assumption.

We don't encrypt nearly enough

The most surprising lesson of the NSA stories is that 20 years after the development of SSL encryption, we're still sending vast amounts of valuable data in the clear.

Even as late as 2014, highly vulnerable client-to-server connections for services like Yahoo Mail were routinely transmitted in cleartext -- meaning that they weren't just vulnerable to the NSA, but also to everyone on your local wireless network. And web-based connections were the good news. Even if you carefully checked your browser connections for HTTPS usage, proprietary extensions and mobile services would happily transmit data such as your contact list in the clear. If you noticed and shut down all of these weaknesses, it still wasn't enough -- tech companies would naively transmit the same data through vulnerable, unencrypted inter-datacenter connections where the NSA could scoop them up yet again.

There is a view in our community that we're doing much better now, and to some extent we may be. But I'm less optimistic. From an attacker's point of view, the question is not how much we're encrypting, but rather, which valuable scraps we're not protecting. As long as we tolerate the existence of unencrypted protocols and services, the answer is still: way too much.

It's the metadata, stupid

Even if we, by some miracle, manage to achieve 100% encryption of communications content, we still haven't solved the whole problem. Unfortunately, today's protocols still leak a vast amount of useful information via session metadata. And we have no good strategy on the table to defend against it.

Examples of metadata leaked by today's protocols include protocol type, port number, and routing information such as source and destination addresses. It also includes traffic characteristics, session duration, and total communications bandwidth. Traffic analysis remains a particular problem: even knowing the size of the files requested by a TLS-protected browser connection can leak a vast amount of information about the user's browsing habits.

Absolutely none of this is news to security engineers. The problem is that there's so little we can do about it. Anonymity networks like Tor protect the identity of endpoints in a connection, but they do so at a huge cost in additional bandwidth and latency -- and they offer only limited protection in the face of a motivated global adversary. IPSec tunnels only kick the can to a different set of trusted components that themselves can be subverted.

'Full take' culture

Probably the most eye-opening fact of the intelligence leaks is the sheer volume of data that intelligence agencies are willing to collect. This is most famously exemplified by the U.S. bulk data collection and international call recording programs -- but for network engineers the more worrying incarnation is "full take" Internet collection devices like TEMPORA.

If we restrict our attention purely to the collection of such data -- rather than how it's accessed -- it appears that the limiting factors are almost exclusively technical in nature. In other words, the amount of data collected is simply a function of processing power, bandwidth and storage. And this is bad news for our future.

That's because while meaningful human communication bandwidth (emails, texts, Facebook posts, Snapchats) continues to increase substantially, storage and processing power increase faster. With some filtration, and no ubiquitous encryption, 'full take' is increasingly going to be the rule rather than the exception.

We've seen the future, and it's not American

Even if you're not inclined to view the NSA as an adversary -- and contrary to public perception, that view is not uniform even inside Silicon Valley -- America is hardly the only intelligence agency capable of subverting the global communications network. Nations like China are increasingly gaining market share in telecommunications equipment and services, especially in developing parts of the world such as Africa and the Middle East.

While it's cheap to hold China out as some sort of boogeyman, it's significant that someday a large portion of the world's traffic will flow through networks controlled by governments that are, at least to some extent, hostile to the core values of Western democracies.

If you believe that this is the future, then the answer certainly won't involve legislation or politics. The NSA won't protect us through cyber-retaliation or whatever plan is on the table today. If you're concerned about the future, then the answer is to finally, truly believe our propaganda about network trust. We need to learn to build systems today that can survive such an environment. Failing that, we need to adjust to a very different world.

by Matthew Green ( at August 16, 2015 07:22 PM

Ansible-cmdb v1.3: a host overview generator for ansible-managed hosts

A few days ago I released ansible-cmdb. Ansible-cmdb takes the output of Ansible's setup module and converts it into a static HTML overview page containing system configuration information. It supports multiple templates and extending information gathered by Ansible with custom data.

The tool was positively received and I got lots of good feedback. This has resulted in v1.3 of ansible-cmdb, which you can download from the releases page.

This is a maintenance release that fixes the following issues:

  • Generated RPM now installs on operating systems with strict Yum (Fedora 22, Amazon AMI).
  • The default templates (html_fancy, txt_table) no longer crash on missing information.
  • Python3 compatibility. (by Sven Schliesing).
  • Disk total and available columns have been deprecated in favour of adding the information to the Disk Usage columns. (by Sven Schliesing).
  • No longer ignore disks smaller than 1Gb, but still ignore disks of total size 0.
  • Minor fixes in the documentation (by Sebastian Gumprich, et al).
  • Better error reporting.

For more information, see the Github page. Many thanks to the bug reporters and contributors!

by admin at August 16, 2015 11:37 AM

August 15, 2015


New Website

We just went live with a new website. The design is based on the style included with Octopress; the new logo and some other important CSS tweaks were contributed by Tony Arcieri. The style is also mobile-friendly, so you can take us with you wherever you go. :) We still need a better “favicon.”

The text still needs more work. As someone on the team pointed out, “a worldwide community of volunteers that use the Internet to communicate, plan, and develop [OpenSSL]” … really?

The online manpages aren’t there yet. Our plan is to have all versions online. But if anyone has any suggestions on how to make pod2html work with our style, post a comment below.

And, more importantly, if you find any broken links, please let us know that, too!

August 15, 2015 02:00 PM

August 14, 2015


Top Ten Books Policymakers Should Read on Cyber Security

I've been meeting with policymakers of all ages and levels of responsibility during the last few months. Frequently they ask "what can I read to better understand cyber security?" I decided to answer them collectively in this quick blog post.

By posting these, I am not endorsing everything they say (with the exception of the last book). On balance, however, I think they provide a great introduction to current topics in digital security.

  1. Cybersecurity and Cyberwar: What Everyone Needs to Know by Peter W. Singer and Allan Friedman
  2. Countdown to Zero Day: Stuxnet and the Launch of the World's First Digital Weapon by Kim Zetter
  3. @War: The Rise of the Military-Internet Complex by Shane Harris
  4. China and Cybersecurity: Espionage, Strategy, and Politics in the Digital Domain by  Jon R. Lindsay, Tai Ming Cheung, and Derek S. Reveron
  5. Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World by Bruce Schneier
  6. Spam Nation: The Inside Story of Organized Cybercrime-from Global Epidemic to Your Front Door by Brian Krebs
  7. Future Crimes: Everything Is Connected, Everyone Is Vulnerable and What We Can Do About It by Marc Goodman
  8. Chinese Industrial Espionage: Technology Acquisition and Military Modernisation by William C. Hannas, James Mulvenon, and Anna B. Puglisi 
  9. Cyber War Will Not Take Place by Thomas Rid
  10. The Practice of Network Security Monitoring: Understanding Incident Detection and Response by Richard Bejtlich (use code NSM101 to save 30%; I prefer the print copy!)


by Richard Bejtlich ( at August 14, 2015 07:28 PM

The Lone Sysadmin

Interesting Dell iDRAC Tricks

Deploying a bunch of machines all at once? Know your way around for loops in shell scripts, or Excel enough to do some basic text functions & autofill? You, too, can set up a few hundred servers in one shot. Here’s some interesting things I’ve done in the recent past using the Dell iDRAC out-of-band […]

The post Interesting Dell iDRAC Tricks appeared first on The Lone Sysadmin. Head over to the source to read the full post!

by Bob Plankers at August 14, 2015 02:58 AM

August 13, 2015


Translating Webhooks with AWS API Gateway and Lambda

Webhooks are great, so many services now support them but I found actually doing anything with them a pain as there are no standards for what goes in them and any 3rd party service you wish to integrate with has to support the particular hooks you are producing.

For instance I want to use SignalFX for my metrics and events but they have very few integrations. A translator could take an incoming hook and turn it into a SignalFX event and pass it onward.

For a long time I’ve wanted to build a translator but never got around to doing it because I did not feel like self hosting it and write a whole bunch of supporting infrastructure. With the release of AWS API Gateway this has become quite easy and really convenient as there are no infrastructure or instances to manage.

I’ll show a bit of a walk through on how I built a translator that sends events to Signal FX. Note I do not do any kind of queueing or retrying on the gateway at present so it’s lossy and best efforts.

AWS Lambda runs stateless functions on demand. At launch it only supported ingesting their own Events but the recently launched API Gateway lets you front it using a REST API of your own design and this made it a lot easier.

For the rest of this post I assume you’re over the basic hurdles of signing up for AWS and are already familiar with the basics, so some stuff will be skipped but it’s not really that complex to get going.

The Code

To get going you need some JS code to handle the translation, here’s a naive method to convert a GitHub push notification into a SignalFX event:

This will be the meat of the of the processing and it includes a bit of code to create a request using the https module which includes the SignalFX authentication header.

Note this creates dimensions to the event that is being sent, I guess you can think of them like some kind of key=val tags for the event. In the Signal FX UI I can select events like this:

And any other added dimension can be used too, events shows up as little diamonds on graphs, so if I am graphing a service using these dimensions I can pick out events that relate to the branches and repositories that influence the data.

This is called as below:

There’s some stuff not shown here for brevity, it’s all in GitHub. The entry point here is handleGitHubPushNotifications, this is the Lambda function that will be run. I can put many different ones in here and in the previous code and share this same zip file across many functions. All I have to do is tell Lambda to run handleGitHubPushNotifications or handleOpsGeniePushNotifications etc. so this is a library of functions. See the next section for how.

Setting up the Lambda functions

We have to create a Lambda function, for now I’ll use the console but you can use terraform for this it helps quite a lot.

As this repo is made up of a few files your only option is to zip it up. You’ll have to clone it and make your own config.js based on the sample prior to creating the zip file.

Once you have it just create a Lambda function which I’ll call gitHubToSFX and choose your zip file as source. While setting it up you have to supply a handler. This is how Lambda finds your function to call.

In my case I specify index.handleGitHubPushNotifications – uses the handleGitHubPushNotifications function found in index.js.

It ends up looking like this:

Once created you can test it right there if you have a sample GitHub commit message.

The REST End Point

Now we need to create somewhere for GitHub to send the POST request to. Gateway works with resources and methods. A resource is something like /github-hook and a method is POST.

I’ve created the resource and method, and told it to call the Lambda function here:

You have to deploy your API – just hit the big Deploy API button and follow the steps, you can create stages like development, staging, production and deploy API’s through such a life cycle. I just went straight to prod.

Once deployed it gives you a URL like and your GitHub hook would be configured to hit .


That’s about it, once you’ve configured GitHub you’ll start seeing events flow through.

Both Lambda and API Gateway can write logs to Cloud Watch and from the JS side you can see do something like console.log(“hello”) and this will show up in the Cloud Watch logs to help with debugging.

I hope to start gathering a lot of translations like these and am still learning Node, so not really sure yet how to make packages or classes but so far this seems really easy to use.

Cost wise it’s really cheap. You’d pay $3.50 per million API calls received on the Gateway and $0.09/GB for the transfer costs, but given the nature of these events this will be negligible. Lambda is free for the first 1 million requests and you’ll pay some tiny amount for the time used. They are both eligible for the free tier too in case you’re new to AWS.

There are many advantages to this approach:

  • It’s very cheap as there are no instances to run, just the requests
  • Adding webhooks to many services is a clickfest hell. This gives me a API that I can change the underlying logic of without updating GitHub etc
  • Today I use SignalFX but it’s event feature is pretty limited, I can move all the events elsewhere on the backend without any API changes
  • I can use my own domain and SSL certs
  • As the REST API is pretty trivial I can later move it in-house if I need, again without changing any 3rd parties – assuming I set up my own domain

I have 2 outstanding issues to address:

  • How to secure it, API Gateway supports headers as tokens but this is not something webhooks tend to support
  • Monitoring it, I do not want to some webhook sender to get in a loop and send 100s of thousands of requests without it going unnoticed

by R.I. Pienaar at August 13, 2015 07:15 PM

League of Professional System Administrators

LOPSA New Board Officers and Focus for the next year

The LOPSA Board held its annual Face to Face meeting where the old board passed on the leadership of LOPSA to the new board.  Your officers for the next year are:

read more

by lopsawebstaff at August 13, 2015 06:28 PM

Steve Kemp's Blog

Making an old android phone useful again

I've got an HTC Desire, running Android 2.2. It is old enough that installing applications such as thsoe from my bank, etc, fails.

The process of upgrading the stock ROM/firmware seems to be:

  • Download an unsigned zip file, from a shady website/forum.
  • Boot the phone in recovery mode.
  • Wipe the phone / reset to default state.
  • Install the update, and hope it works.
  • Assume you're not running trojaned binaries.
  • Hope the thing still works.
  • Reboot into the new O/S.

All in all .. not ideal .. in any sense.

I wish there were a more "official" way to go. For the moment I guess I'll ignore the problem for another year. My nokia phone does look pretty good ..

August 13, 2015 02:44 PM

August 12, 2015

Introducing ansible-cmdb: a host overview generator for ansible-managed hosts

For those of you that are using Ansible to manage hosts, you may have noticed you can use the setup module to gather facts about the hosts in your inventory:

$ ansible -m setup --tree out/ all
$ ls out
$ head out/ 
    "ansible_facts": {
        "ansible_all_ipv4_addresses": [
        "ansible_all_ipv6_addresses": [
        "ansible_architecture": "x86_64", 
        "ansible_bios_date": "12/01/2006",

The setup module in combination with the --tree option produces a directory of JSON files containing facts about ansible-managed hosts such as hostnames, IP addresses, total available and free memory, and much more.

I wrote ansible-cmdb to take that output and generate an user-friendly host overview / CMDB (Configuration Management Database) HTML page. Usage is simple:

$ ansible -m setup --tree out/ all   # generate JSON output facts
$ ansible-cmdb out/ > cmdb.html      # generate host-overview page

Here's an example of what it produces.

And here's a screenshot:


It can read your hosts inventory and gather variable values from it, which can be used in the templates that produce the output. You can also extend the gathered facts easily with your own facts by manually creating or generating additional output directories containing JSON files. This even allows you to manually define hosts which are not managed by Ansible.

Ansible-cmdb is template-driven, which means it's rather easy to modify the output. The output is generated using Mako templates

I've just released v1.2. Packages are available in source, Debian/Ubuntu and Redhat/Centos formats. 

For more information, see the Github page. I hope you like it!

by admin at August 12, 2015 01:00 PM


The questions to ask before moving to git

Or, a post I'd never thought I'd make seeing as I'm a sysadmin.

But it seems I'm the senior git expert in my team, so I'm making it. So odd.

There are a series of questions you should ask among your team before moving a repo over to git. Git is a hell of a toolbox, and like all toolboxes there are nearly infinite ways of using it. There is no one true way, only ways that are better for you than others. These are a series of questions to help you figure out how you want to use it, so you can be happier down the road.

Q: How do you use the commit-log?

History is awesome. Looking back five years in the code repository to figure out WTF a past developer was thinking about writing that bit of spaghetti code is quite useful if that commit includes something like, "found weird-ass edge case in glib, this is the workaround until they get a fix." That's actionable. Maybe it's even tied to a bug number in the bug tracking system, or a support ticket.

Do you ever look through the history? What are you looking for? Knowing this allows you to learn what you want out of your source-control.

Q: What is the worth of a commit?

A commit in Git is not the same thing as in SVN, Fog, or ClearCase. In some, a commit, or checkin, is a pretty big thing. It takes reviews, or approvals before it can be made.

This question is there to get you thinking about what a commit is. Commits in git are cheap, that changes things. Knowing that you will be facing more of then than you had in the past will help guide you in the later questions.

Q: Is every commit sacred, or you do you value larger, well documented commits more?

Practically everyone I know has made a commit with the message of 'asdf'. If you're grinding on a stupid thing, it may take you 19 commits to come up with the two lines of code that actually work. In five years, when you come back to look at that line of code, the final commit-message on those lines might be '

a1bd0809 maybe this will work

Not exactly informative.

bdc8671a Reformat method calls to handle new version of nokogiri

That is informative.

Most projects value more informative commits over lots of little, iterative ones. But your team may be different. And may change its mind after experience has been had.

Q: Should new features be all in one commit, or in a few modular commits?

Some features are quite large. So large, that rebasing them into a single commit leads to a diff of hundreds of lines. Such a large feature means that the history on those files will be slathered with the same initial-feature-commit with no context for why it is that way.

Is that good enough? Mabe it is, maybe you're more interested in the hotfix commits that are fixing bugs and explain non-intuitive behavior and workaround. Maybe it isn't, and you need each sub-feature in its own. Or maybe you want every non-fixup commit.

This is where your approach to the history really informs your decision. If you know how you deal with the past, you will be better able to put process in place to be happier with your past self.

Once you've thought about these questions and your answers to them, you'll be better able to consider the deeper problem of branching strategy. Git is notoriously lacking in undo features, at least in shared repos, so getting this out of the way early is good.

by SysAdmin1138 at August 12, 2015 12:39 AM

August 10, 2015

Steve Kemp's Blog

A brief look at the weed file store

Now that I've got a citizen-ID, a pair of Finnish bank accounts, and have enrolled in a Finnish language-course (due to start next month) I guess I can go back to looking at object stores, and replicated filesystems.

To recap my current favourite, despite the lack of documentation, is the Camlistore project which is written in Go.

Looking around there are lots of interesting projects being written in Go, and so is my next one the seaweedfs, which despite its name is not a filesystem at all, but a store which is accessed via HTTP.

Installation is simple, if you have a working go-lang environment:

go get

Once that completes you'll find you have the executable bin/weed placed beneath your $GOPATH. This single binary is used for everything though it is worth noting that there are distinct roles:

  • A key concept in weed is "volumes". Volumes are areas to which files are written. Volumes may be replicated, and this replication is decided on a per-volume basis, rather than a per-upload one.
  • Clients talk to a master. The master notices when volumes spring into existance, or go away. For high-availability you can run multiple masters, and they elect the real master (via RAFT).

In our demo we'll have three hosts one, the master, two and three which are storage nodes. First of all we start the master:

root@one:~# mkdir /
root@one:~# weed master -mdir / -defaultReplication=001

Then on the storage nodes we start them up:

root@two:~# mkdir /data;
root@two:~# weed volume -dir=/data -max=1  -mserver=one.our.domain:9333

Then the second storage-node:

root@three:~# mkdir /data;
root@three:~# weed volume -dir=/data -max=1 -mserver=one.our.domain:9333

At this point we have a master to which we'll talk (on port :9333), and a pair of storage-nodes which will accept commands over :8080. We've configured replication such that all uploads will go to both volumes. (The -max=1 configuration ensures that each volume-store will only create one volume each. This is in the interest of simplicity.)

Uploading content works in two phases:

  • First tell the master you wish to upload something, to gain an ID in response.
  • Then using the upload-ID actually upload the object.

We'll do that like so:

laptop ~ $ curl -X POST http://one.our.domain:9333/dir/assign

client ~ $ curl -X PUT -F file=@/etc/passwd,06c3add5c3

In the first command we call /dir/assign, and receive a JSON response which contains the IPs/ports of the storage-nodes, along with a "file ID", or fid. In the second command we pick one of the hosts at random (which are the IPs of our storage nodes) and make the upload using the given ID.

If the upload succeeds it will be written to both volumes, which we can see directly by running strings on the files beneath /data on the two nodes.

The next part is retrieving a file by ID, and we can do that by asking the master server where that ID lives:

client ~ $ curl http://one.our.domain:9333/dir/lookup?volumeId=1,06c3add5c3

Or, if we prefer we could just fetch via the master - it will issue a redirect to one of the volumes that contains the file:

client ~$ curl http://one.our.domain:9333/1,06c3add5c3
<a href=",06c3add5c3">Moved Permanently</a>

If you follow redirections then it'll download, as you'd expect:

client ~ $ curl -L http://one.our.domain:9333/1,06c3add5c3

That's about all you need to know to decide if this is for you - in short uploads require two requests, one to claim an identifier, and one to use it. Downloads require that your storage-volumes be publicly accessible, and will probably require a proxy of some kind to make them visible on :80, or :443.

A single "weed volume .." process, which runs as a volume-server can support multiple volumes, which are created on-demand, but I've explicitly preferred to limit them here. I'm not 100% sure yet whether it's a good idea to allow creation of multiple volumes or not. There are space implications, and you need to read about replication before you go too far down the rabbit-hole. There is the notion of "data centres", and "racks", such that you can pretend different IPs are different locations and ensure that data is replicated across them, or only within-them, but these choices will depend on your needs.

Writing a thin middleware/shim to allow uploads to be atomic seems simple enough, and there are options to allow exporting the data from the volumes as .tar files, so I have no undue worries about data-storage.

This system seems reliable, and it seems well designed, but people keep saying "I'm not using it in production because .. nobody else is" which is an unfortunate problem to have.

Anyway, I like it. The biggest omission is really authentication. All files are public if you know their IDs, but at least they're not sequential ..

August 10, 2015 01:29 PM

hENC version 3 released

github-logo Today I am releasing the version 3 of hENC, the radically simple hierarchical External Node Classifier (ENC) for CFEngine (version 2 was released at the end of May and added support for data containers).

This version adds new features and bug fixes, namely:

  • implemented !COMMANDS: a ! primitive is added to specify commands; three commands exist currenty: !RESET_ACTIVE_CLASSES to make hENC forget about any class that was activated up to that point, !RESET_CANCELLED_CLASSES ditto for cancelled classes, and !RESET_ALL_CLASSES that makes hENC forget about any class that was activated or cancelled;
  • fixed, so that it is possible to run the henc module more than once during the same agent run;
  • added a Changelog;
  • improved tests: tests have been added for the new features and the whole test suite has been improved to support the TAP protocol; for example, it’s now it’s possible to use the prove utility to verify if hENC works correctly on your system before trying the installation.

See the README and Changelog for more information.

Tagged: cfengine, Configuration management, Github, henc

by bronto at August 10, 2015 12:29 PM

August 09, 2015


UbuConLA 2015 in Lima

This week I had the honor of joining a couple hundred free software enthusiasts at UbuCon Latin America. I’d really been looking forward to it, even if I was a bit apprehensive about the language barrier, and the fact that mine was the only English talk on the schedule. But those fears melted away as the day began on Friday morning and I found myself loosely able to follow along with sessions with the help of slides, context and my weak understanding of Spanish (listening is much easier than speaking!).

The morning began by meeting a couple folks from Canonical and a fellow community member at the hotel lobby and getting a cab over to the venue. Upon arrival, we were brought into the conference speaker lounge to settle in before the event. Our badges had already been printed and were right there for us, and bottles of water available for us, it was quite the pleasant welcome.

José Antonio Rey kicked off the event at 10AM with a welcome, basic administrative notes about the venue, a series of thanks and schedule overview. Video (the audio in the beginning sounds like aliens descending, but it gets better by the end).

Immediately following him was a keynote by Pablo Rubianes, a contributor from Uruguay who I’ve known and worked with in the Ubuntu community for several years. As a member of the LoCo Council, he had a unique view into development and construction of LoCo (Local/Community) teams, which he shared in this talk. He talked some about how LoCos are organized, gave an overview of the types of events many of them do, like Ubuntu Hours, Global Jams and events in collaboration with other communities. I particularly enjoyed the photos he shared in his presentation. He left a lot of time for questions, which was needed as many people in the audience had questions about various aspects of LoCo teams. Also, I enjoyed the playful and good humored relationship they have with the title “LoCo” given the translation of the word into Spanish. Video.

My keynote was next, Building a Career in Free and Open Source Software (slides, English and Spanish). Based on audience reaction, I’m hopeful that a majority of the audience understood English well enough to follow along. For anyone who couldn’t, I hope there was value found in my bi-lingual slides. I had some great feedback following my talk both in person and on Twitter. Video (in English!).

Thanks to Pablo Rubianes for the photo (source)

For all the pre-conference jokes about a “cafeteria lunch” I was super impressed with my lunch yesterday. Chicken and spiced rice, some kind of potato-based side and a dessert of Chicha Morada pudding… which is what I called it until I learned the real name, Mazamorra Morada, a purple corn pudding that tastes like the drink I named it after. Yum!

After lunch we heard from Naudy Villaroel who spoke about the value of making sure people of all kinds are included in technology, regardless of disability. He gave an overview of several accessibility applications available in Ubuntu and beyond, including the Orca screen reader, the Enable Viacam (eViacam) tool for controlling the mouse through movements on camera and Dasher which allows for small movements to control words that are displayed through algorithms that anticipate words and letters the operator will want to use, and makes it easy to form them. He then went on to talk about other sites and tools that could be used. Video.

Following Naudy’s talk, was one by Yannick Warnier, president of Chamilo, which produces open source educational software. His talk was a tour of how online platforms, both open source and hosted (MOOC-style) have evolved over the past couple decades. He concluded by speculating far into the future as to how online learning platforms will continue to evolve and how important education will continue to be. Video. The first day concluded with a duo of talks from JuanJo Ciarlante, the first about free software on clouds (video… and ran over so continued in next link…) and a second that covered some basics around using Python to do data crunching, including some of the concepts around Map Reduce type jobs and Python-based libraries to accomplish it (video, which includes the conclusion of the cloud talk, the last half is about Python).

The evening was spent with several of my fellow speakers at La Bistecca. I certainly can’t say I haven’t been eating well while I’ve been here!

I also recommend reading Jose’s post about the first day, giving you a glimpse into the work he’s done to organize the conference here: UbuConLA 2015: The other side of things. Day 1.

And with that, we were on to day 2!

The day began at 10AM with a talk about Snappy by Sergio Schvezov. I was happy to have read a blog post by Ollie Ries earlier in the week that walked through all the Snappy/core/phone related names that have been floating around, but this talk went over several of the definitions again so I’m sure the audience was appreciative to get them straightened out. He brought along a BeagleBone and Ubuntu tablet that he did some demos on as he deployed Ubuntu Core and introduced Snapcraft for making Snappy packages. Video.

Following his talk was one by Luis Michael Ibarra in a talk about the Linux container hypervisor, LXD. I learned that LXD was an evolution of lxc-tools, and in his talk he dug through the filesystem and system processes themselves to show how the containers he was launching worked. Unfortunately his talk was longer than his slot, so he didn’t get through all his carefully prepared slides, so hopefully they’ll be published soon. Video.

Just prior to lunch, we enjoyed a talk by Sebastián Ferrari about Juju where he went through the background of Juju, what it’s for and where it fits into the deployment and orchestration world. He gave demos of usage and the web interface for it on both Amazon and Google Compute Engine. He also provided an introduction to the Juju Charm Store where charms for various applications are shared and shared the JuJu documentation for folks looking to get started with Juju. Video.

After lunch the first talk was by Neyder Achahuanco who talked about building Computer Science curriculum for students using tools available in Ubuntu. He demonstrated Scratch, Juegos de Blockly (Spanish version of Blockly Games), (which is in many languages, see bottom right of the site) and MIT App Inventor. Video).

Break, with Ubuntu and Kubuntu stickers!

As the afternoon continued, Pedro Muñoz del Río spoke on using Ubuntu for a platform for data analysis. Video. the Talks concluded with Alex Aragon who gave an introduction to 3d animation with Blender where he played the delightful Monkaa film. He then talked about features and went through various settings. Video.

Gracias to all the organizers, attendees and folks who made me feel welcome. I had a wonderful time! And as we left, I snagged a selfie with the flags flying outside the University. For what? Jose picked them out upon learning which countries people would be flying in from, the stars and stripes were flying for me!

More photos from UbuConLA here:

August 09, 2015 03:13 AM

August 08, 2015

The Lone Sysadmin

10 Years

Ten years ago I wrote the first post on this blog. 3:43 AM. I’m a late night kinda guy, I guess. Actually, I probably came home from a bar, installed WordPress 1.5.1, and started writing. Ten years seems like an awfully long time ago. So much has changed in my life. I like my job, […]

The post 10 Years appeared first on The Lone Sysadmin. Head over to the source to read the full post!

by Bob Plankers at August 08, 2015 09:43 AM


Effect of Hacking on Stock Price, Or Not?

I read Brian Krebs story Tech Firm Ubiquiti Suffers $46M Cyberheist just now. He writes:

Ubiquiti, a San Jose based maker of networking technology for service providers and enterprises, disclosed the attack in a quarterly financial report filed this week [6 August; RMB] with the U.S. Securities and Exchange Commission (SEC). The company said it discovered the fraud on June 5, 2015, and that the incident involved employee impersonation and fraudulent requests from an outside entity targeting the company’s finance department.

“This fraud resulted in transfers of funds aggregating $46.7 million held by a Company subsidiary incorporated in Hong Kong to other overseas accounts held by third parties,” Ubiquiti wrote. “As soon as the Company became aware of this fraudulent activity it initiated contact with its Hong Kong subsidiary’s bank and promptly initiated legal proceedings in various foreign jurisdictions. As a result of these efforts, the Company has recovered $8.1 million of the amounts transferred.”

Brian credits Brian Honan at CSO Online, with noticing the disclosure yesterday.

This is a terrible crime that I would not wish upon anyone. My interest in this issue has nothing to do with Ubiquiti as a company, nor is it intended as a criticism of the company. The ultimate fault lies with the criminals who perpetrated this fraud. The purpose of this post is to capture some details for the benefit of analysis, history, and discussion.

The first question I had was: did this event have an effect on the Ubiquiti stock price? The FY fourth quarter results were released at 4:05 pm ET on Thursday 6 August 2015, after the market closed.

The "Fourth Quarter Financial Summary: listed this as the last bullet:

"GAAP net income and diluted EPS include a $39.1 million business e-mail compromise ("BEC") fraud loss as disclosed in the Form 8-K filed on August 6, 2015"

I assume the Form 8-K was published simultaneously, with earnings.

Next I found the following in this five day stock chart.

5 day UBNT Chart (3-7 August 2015)

You can see the gap down from Thursday's closing price, on the right side of the chart. Was that caused by the fraud charge?

I looked to see what the financial press had to say. I found this Motley Fool article titled Why Ubiquiti Networks, Inc. Briefly Fell 11% on Friday, posted at 12:39 PM (presumably ET). However, this article had nothing to say about the fraud.

Doing a little more digging, I saw Seeking Alpha caught the fraud immediately, posting Ubiquiti discloses $39.1M fraud loss; shares -2.9% post-earnings at 4:24 PM (presumably ET).  They noted that "accounting chief Rohit Chakravarthy has resigned." I learned that the company was already lacking a chief financial officer, so Mr. Chakravarthy was filling the role temporarily. Perhaps that contributed to the company falling victim to the ruse. Could Ubiquiti have been targeted for that reason?

I did some more digging, but it looks like the popular press didn't catch the issue until Brian Honan and Brian Krebs brought attention to the fraud angle of the earnings release, early today.

Next I listened to the archive of the earnings call. The call was a question-and-answer session, rather than a statement by management followed by Q and A. I listened to analysts ask about head count, South American sales, trademark names, shipping new products, and voice and video. Not until the 17 1/2 minute mark did an analyst ask about the fraud.

CEO Robert J. Pera said he was surprised no one had asked until that point in the call. He said he was embarrassed by the incident and it reflected "incredibly poor judgement and incompetence" by a few people in the accounting department.

Finally, returning to the stock chart, you see a gap down, but recovery later in the session. The market seems to view this fraud as a one-time event that will not seriously affect future performance. That is my interpretation, anyway. I wish Ubiquiti well, and I hope others can learn from their misfortune.

Update: I forgot to add this before hitting "post":

Ubiquiti had FY fourth quarter revenues of $145.3 million. The fraud is a serious portion of that number. If Ubiquiti had earned ten times that in revenue, or more, would the fraud have required disclosure?

The disclosure noted:

"As a result of this investigation, the Company, its Audit Committee and advisors have concluded that the Company’s internal control over financial reporting is ineffective due to one or more material weaknesses."

That sounds like code for a Sarbanes-Oxley issue, so I believe they would have reported anyway, regardless of revenue-to-fraud proportions.

by Richard Bejtlich ( at August 08, 2015 12:36 AM

August 07, 2015

Standalone Sysadmin

Ad Astra Per Aspera – Leaving Boston

northeasterneduI really like working at Northeastern University, which is why I’m sad that I’m going to be leaving. On the other hand, life occasionally presents an opportunity to you that can’t ignore. This is one of those occasions.

A few months ago, I was sitting in a small room full of sysadmins planning LISA’15 when I mentioned, almost out of nowhere, that there was one company in the world that I would kill to work at. As luck would have it, my friend sitting next to me said, “Really? Because I know a guy. Want me to email him for you?” and I said, “Um, yes, please. ” Thus a story began that included numerous phone screenings, flying out to Los Angeles, and an all-day array of in-person interviews, the net result being that I am leaving Boston, moving to LA, and going to work…for Space Exploration Technologies Corporation, otherwise known as SpaceX. Yes, THAT SpaceX.



At SpaceX, I’m going to be a Linux System Administrator, and from the sounds of it, I’ll be splitting my time between “normal” infrastructure stuff and helping to define a DevOps role with the Flight Software team who write the software that sends the rocket and Dragon capsule up to the Space Station. It’s…pretty difficult to overstate how excited I am.


I imagine that it will take a while to figure out what I’m allowed to write about, but the whole team was very enthusiastic about my visibility in the SysAdmin space, and they seemed to enjoy my blog and the fact that I took part in the community, so I don’t think anything there will change. I’m just really happy to get the chance to do this, for a company with a mission like SpaceX. It’s an incredible opportunity, and I feel very fortunate.

So here we go, on a brand new adventure. I’m sad to be leaving my friends in Boston, but I’ll be back soon – I mean heck, LISA’16 is in Boston, so it’ll be like a homecoming, right? Until then, the sky is the limit! Keep reading!



by Matt Simmons at August 07, 2015 12:30 PM

Evaggelos Balaskas

PowerDNS Remote Backend

One of the great features that PowerDNS has, is the concepts of ‘backends’.

Backends give you the ability to choose the datastore you would like to save (or not) your dns data. If you are looking to migrate from another dns server (lets say bind ics) with bind zone files support, then you can choose the bind backend, copy the files and voila !

PowerDNS can also support multiple backends. So you can build/test your “new” infrastructure without compromise any existing data structure or as the consultants love to say: “With no-downtime!” Another approach is that you can add support for provisioning automate mechanism or whatever else you can think of !

A very good example of Pipe Backend is the PowerDNS Dynamic Reverse script that @kargig has modified to support reverse ipv6 responses (amazing, right ?).

I have a few (half–baked) ideas that I would like to implement with PowerDNS and I was looking on Remote Backend. It took me some time to understand the logic behind this (as I am not a developer, nor I will ever be!) and create a proof of concept script.

So this is my initial script, that I would like to share:
pdns remote - pipe

It doesnt do anything (yet), just sends everything to your syslog (/var/log/messages) for debugging.

The key to success is this quote:

You must always reply with JSON hash with at least one key, ‘result’

Tag(s): PowerDNS

August 07, 2015 11:09 AM

The Lone Sysadmin

Three Thoughts on the Nutanix & StorageReview Situation

I’ve watched the recent dustup between VMware and Nutanix carefully. It’s very instructive to watch how companies war with each other in public, and as a potential customer in the hyperconverged market it’s nice to see companies go through a public opinion shakedown. Certainly both VMware and Nutanix tell stories that seem too good to […]

The post Three Thoughts on the Nutanix & StorageReview Situation appeared first on The Lone Sysadmin. Head over to the source to read the full post!

by Bob Plankers at August 07, 2015 06:12 AM

August 05, 2015

The Tech Teapot

A retrospective on Dyna Project version 0.1

Whilst version 0.1 of the Dyna Project isn’t quite finished, I thought it would make sense to take stock before work starts on version 0.2.

But first some introductions would probably be helpful.

What is the Dyna Project?

For a lot of years I’ve been interested in constraint satisfaction problems and how to solve them.

The Dyna Project is my latest attempt to create a tool for solving constraint type problems in an accessible way. I want constraint type problems to be as solvable as financial modelling is with a spreadsheet.

A retrospective on version 0.1

I’ve been working part time on the Dyna Project since 7th April this year. The project has received 122 commits so far, with approximately 5K lines of C# code. My plan was to build the simplest graphical modeller of constraint type problems possible. I think I have achieved that, it could not be much simpler.

Whilst the solution is very simple, there is the kernel of the design already. There is a place to model your problem, a mechanism for solving it and another mechanism to display the solution. The only element missing is the ability to design how the solution will be displayed.

I can’t say I like anything about the model building interface or design. It does work, but it is very painful to use. The idea of using different shapes for the variables, domains and constraints doesn’t work at all. The connectors add a lot of ceremony but don’t add much in the way of clarity. Most work in the 0.2 release will need to be used resolving the model building interface. Without a workable model interface, there’s not much point to the rest of the program.

The project name will be changing before the 0.2 release. The name clashes with another project on GitHub so I think it only polite to find another one. No idea what the new name will be yet. I am open to suggestions. :)

Please don’t use the project yet, it is some distance away from being usable. I know it’s buggy, and have no intention of fixing the bugs because much of the code is going to change for version 0.2.


The design of version 0.1 was only ever intended as a place marker. In that and that alone I think it has succeeded. The outline of the program is in there, all of those elements need considerable improvement in the next few releases.

by Jack Hughes at August 05, 2015 03:46 PM

August 04, 2015

Debian Administration

Validating SPF and DKIM at SMTP-time with exim

In our recent articles we've discussed creating SPF-records to avoid spoofed mails, and the creation and setup for DKIM-signing emails, for a similar purpose. Here we'll look at the other side of the coin; performing DKIM and SPF testing on your incoming email.

by Steve at August 04, 2015 07:58 AM

August 03, 2015

Debian Administration

Tying together SPF and DKIM with DMARC

When it comes to increasing deliverabiity of email, and preventing spoofed/forged emails the preferred solution these days is DMARC, which allows the use of SPF and DKIM to be enforced for domains in a unified manner.

by Steve at August 03, 2015 07:35 PM

Anton Chuvakin - Security Warrior

Monthly Blog Round-Up – July 2015

Here is my next monthly "Security Warrior" blog round-up of top 5 popular posts/topics this month:
  1. Why No Open Source SIEM, EVER?” contains some of my SIEM thinking from 2009. Is it relevant now? Well, you be the judge.  Current popularity of open source log search tools, BTW, does not break the logic of that post. Succeeding with SIEM requires a lot of work, whether you paid for the software, or not. That – and developing a SIEM is much harder than most people think  [291 pageviews]
  2. Top 10 Criteria for a SIEM?” came from one of my last projects I did when running my SIEM consulting firm in 2009-2011 (for my recent work on evaluating SIEM, see this document) [133 pageviews]
  3. My classic PCI DSS Log Review series is always popular! The series of 18 posts cover a comprehensive log review approach (OK for PCI DSS 3.0 as well), useful for building log review processes and procedures , whether regulatory or not. It is also described in more detail in our Log Management book and mentioned in our PCI book (just out in its 4th edition!) [120+ pageviews to the main tag]
  4. Simple Log Review Checklist Released!” is often at the top of this list – the checklist is still a very useful tool for many people. “On Free Log Management Tools” is a companion to the checklist (updated version) [114 pageviews]
  5. “New SIEM Whitepaper on Use Cases In-Depth OUT!” (dated 2010) presents a whitepaper on select SIEM use cases described in depth with rules and reports [using now-defunct SIEM product]; also see this SIEM use case in depth and this for a more current list of popular SIEM use cases. [62 pageviews of total 4862 pageviews to all blog pages]
In addition, I’d like to draw your attention to a few recent posts from my Gartner blog:

Current maverick research on AI/smart machines risks:
Past research on cloud security monitoring:
Past research on security analytics:
Miscellaneous fun posts:

(see all my published Gartner research here)
Also see my past monthly and annual “Top Popular Blog Posts” – 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014.
Disclaimer: most content at SecurityWarrior blog was written before I joined Gartner on Aug 1, 2011 and is solely my personal view at the time of writing. For my current security blogging, go here.

Previous post in this endless series:

by Anton Chuvakin ( at August 03, 2015 02:40 PM

Standalone Sysadmin

Stop Hating Your Work

I love meeting people at SysAdmin events. Having a blog that people read does mean that people have mostly heard all of my best stories, but it’s still fun getting to know new people and hearing what they’ve been working on. The single thing I hear most often is a question, and the question is, “Don’t you sleep?”

Time and time again, people will read my blog, see me making things, or doing things, or organizing, or whatever, and internally, they compare that to what they do, and they feel like they aren’t doing enough, or as much as I am.

Can I let you in on a secret? I feel like I do crap work most of the time. And I compare myself to others, and to their work, and I feel like what I do is often bad, sub-par, and not worthy.

Do you ever see something that just speaks to your soul? I saw a Tweet, of all things, that did that to me last year. Here it is:

The image from that post features the very first Iron Man suit from Tales of Suspense #39 in 1959, which Tony Stark built in a cave, with a box of scraps. It worked…to a point, but it wasn’t long before it got upgraded and replaced. If you’ve seen the first Iron Man
movie starring Robert Downey Jr, then this will all sound pretty familiar, because it was recreated in film.

It feels sort of childish to admit in an open forum like this, but the story of Tony Stark creating Iron Man is actually really inspirational to me. I like making things. I like building, and doing, and I really, really hate just about everything I create. Especially the early stuff, and Tony embodies the concept of continuous development and iterative improvement that are so vital to making things in 2015. So I try to learn from it, and in my spare time, I try to figure out how repulsor beams work on pure electrical charge.

Earlier this year, I decided that I was going to go to Boston Comic Con for the second year in a row. When I checked out the website, I couldn’t believe my eyes – along with the normal array of comics celebs, Boston was going to be playing host to none other than STAN LEE!

If you don’t know the name Stan Lee, you probably know the characters that he’s made – Spiderman, The X-Men, The Incredible Hulk, Daredevil, Thor, and yes, Iron Man. When I saw that Stan Lee was going to be signing autographs, I knew I had to get one, but the only question was…what would I get signed?

I could always go get a relatively rare Iron Man comic and have him sign that. But none of the individual comics meant as much to me as the character itself. What would be perfect is if I could get that picture from Alexis’s picture above signed, but it’s a PNG, and the quality didn’t really lend itself to blowing up. After thinking for a few minutes, I realized, I didn’t have to use the picture above – I could just recreate it. So I did!

It took me a few hours to get it to the point where I thought it would be acceptable, and fittingly, it isn’t perfect, but here’s the final version that I made:

Click the image above to get the full-sized image. If you want to print your own (don’t sell this – Iron Man is the property of Marvel), you can download the EPS in glorious 41MB fashion from this link.

So yesterday, I visited Comic Con, stood in line for hours, and got to (very briefly) meet Stan Lee, who laughed as he signed his name to my new poster:


I actually printed out two versions – one to keep at work, and this signed one, which I’ll keep at home. Both of them will remind me that, even though I’m probably not happy with the state of whatever I’m working on at the moment, I shouldn’t listen to the negative voices in my head telling me to quit because it isn’t good enough. Thanks Stan!

by Matt Simmons at August 03, 2015 12:30 PM