Geekyness

quick awk trick

published on
While managing a Bacula environment, I occasionally found it useful to find savesets that were still on the Storage Node beyond the life of our retention policy. Since Bacula does not actually erase old volumes if the client next contacts the server again, they hang around and it usually requires manual intervention. find . -mtime +30d -print | xargs du | awk '{ sum += $1;count++ }END{print (sum/1024/1024),count}'

running gitorious on freebsd

published on
Gitorious has some documentation on a local install for Ubuntu, RHEL, and Debian, which turned out to be slightly outdated. Now that they have leveraged the Bundler GEM manager, the installation process is much simpler. FreeBSD is by far my platform of choice when it comes to flexible and complex open source applications, and it was a cinch to reuse the Linux documentation. Aside from having to re-write the init scripts, the entire process was very easy, and I got to try out a few different web servers for the heck of it. Read More...

bacula framework on github

published on
I’m pretty happy with the Bacula environment I’ve created. It has gone through a few iterations, and I’ve learned a lot since I started using it a few years ago. I think its only appropriate to share the evolution of my environment with as many people as possible, and I hope it can help save new bacula administrators some time. Enough of the preamble, here is my github project page: https://github. Read More...

atlassian tools and nginx

published on
Atlassian has a wonderful set of tools. I’ve been using Jira with Fisheye for the last few months, but I recently came across Stash. Stash is a nice git repository and project manager than provides easy access to creating new Git repo’s with User and Group access controls. I’m considering phasing out FishEye in favor of Stash. I only wanted FishEye as a source code browser, that integrated issues tracking. Read More...

puppet module to update freebsd passwd has

published on
LinkedIn’s users database was leaked, and while passwords were not stored in plain text, they were hashed with the MD5 algorithm. Not salted, just hashed. MD5 is no longer as secure as it once was, not with all of these GPU’s lying around. Well, the FreeBSD community has not let this go unnoticed. First off, FreeBSD has defaulted to MD5 for a while, but it has had support for Blow-Fish, and (this is new to me) SHA-256 and SHA-512. Read More...

poor samba performance

published on
If you have google’d for smb.conf settings to max your 1Gb (or 10Gb) ethernet based network and you still cannot seem to get beyond 30MB/sec, make sure you have this option: socket options = SO_RCVBUF=8192 SO_SNDBUF=8192 Either commented it out, or set the buffer size to 128K (131072). Or larger. I was helping a friend of mine troubleshoot his ZFS + Samba environment. He was running FreeBSD 9.0 and Samba 3. Read More...

zfs and acls with samba

published on
I’ve been using ZFS on FreeBSD since it was first made available in 7.0, and it was not until FreeBSD 8.2 when NFSv4 ACL’s were implemented. At $oldJob, I had a list of “To do’s”, really cool things like: “Integrate Puppet into CMDB”, “Rewrite the Bacula addClient script”, “Build a MooseFS Cluster”… At the top though, I had: Implement ACL’s on ZFS/Samba fileserver Actually, come to thing of it, I think the top of my list was: Read More...

mailman with nginx on freebsd

published on
I like Nginx a lot. Not because I’m some sort of hipster sysadmin either. I like it because it is small, fast, and as a FreeBSD port, it compiles and updates quickly. What I also like is the separation of services and processes. For example, if I want to run a PHP script, I don’t load “mod_php” like you would with Apache. Instead, you have a PHP processor, like php-fpm, running (on localhost, or, another server that only processes PHP scripts). Read More...

xen and convirture

published on
My previous experience with a large virtual “enterprise” environment was with VMWare’s ESXi and vSphere. Performance wise, I was always fairly happy with ESXi knowing that it was a virtual machine of course. There were a few issues I recall: silent data corruption (fsck’ing Linux and FreeBSD volumes would reveal inconsistent filesystem information, but there were never any errors reported to the VM) Live migrations were not always stable. In fact, most of the time it would result in an unexpected shutdown I doubt it was solely VMWares fault, as it could have been a series of mis-configurations and poor implementation descisions. Read More...

skynet began in a kitchen

published on
Over the past year or so, Caralyne has become increasingly interested in scientastical things like computers, Minecraft, Robots, Mythbusters, and specifically, Mythbusters! I’ve been using our TV for my own nefarious deeds, unbeknownst to Caralyne, as an educational tool in science and skepticism. Between Good Eats, Mythbusters and a few PBS clips of Neil Degrasse Tyson being awesome, something was bound to stick. Also, as a daughter and child of mine, she is constantly seeking my affection and approval… which I am not above exploiting these features she no doubt got from me, so I give her a LOT of positive feedback and attention when we are watching these shows. Read More...

using couchdb with puppet and bacula

published on
On aspect that I was never happy with the Bacula environment I built while at LLNL was the fact that I could no look up certain values for each client. Values like: * Passwords * Storage Devices * Certificates (if you are using Encryption) Well, over the past few week’s I’ve been able to work around this problem by storing additional information in a CouchDB DB. It is not the ideal solution, but it is a start and I’m okay with that. Read More...

hello bay photo lab

published on
todo From one Lab to another After 9.5 years with one employer (LLNL), I joined Bay Photo Lab in Santa Cruz. This has brought on many changes, not just a career change but a significant change in my way of life. First off, I’m renting a room until my family gets down here. I have this weird double life now, where during the week I live alone in a small room with no heating, and then I go back home on the weekend. Read More...

goodbye llnl

published on
todo Well, after about 9 and a half years with Lawrence Livermore National Laboratory 11⁄09 was my last day. Now, I wasn’t the only admin besides Jenny (thats the aquilino1@llnl.gov email you see there), but she was my closest friend and peer while I was there. There was a little poetic license there, but it was accurate. It is hard to quantify the emotions about leaving LLNL. Read More...

freebsd training

published on
At work I ran a 5 part FreeBSD Administration training course for the Unix team. I enjoyed it a lot, because I really like to share information. It is especially rewarding when it is something I have taken a great interest in, like the FreeBSD Operating System. The design of the course was simple, I did my best to fill in the gap that a Solaris or Linux administrator might have. Read More...

bacula in the enterprise part 2

published on
Software As mentioned many times, this is a FreeBSD based environment. Some good sysinfo output below: Operating system release: FreeBSD 8.2-RELEASE OS architecture: amd64 Kernel build dir location: /usr/obj/usr/src/sys/GENERIC Currently booted kernel: /boot/kernel/kernel Currently loaded kernel modules (kldstat(8)): zfs.ko opensolaris.ko Bootloader settings for the Director/Database node: The /boot/loader.conf has the following contents: kern.ipc.semmni=1024 kern.ipc.semmns=2048 kern.ipc.semmnu=1024 All of the storage nodes and the director are running a GENERIC kernel with very few system tweaking. Read More...

bacula in the enterprise part 1

published on
I’ve been using Bacula, the open source backup software, for over a year now. Things have been going well, and I would like to dedicate a post or two to the environment I built. Background Over a year ago, I took it upon myself to replace a single Legato Networker server with Bacula. One of our collaborators had decided to ship us (for no reason at all really, I think they were cleaning out their data center) a Sun X4200 AMD server, and two StorageTek/Sun NAS servers. Read More...

using duplicity

published on
A while ago, I posted about how I backup my server with Duplicity to Amazon’s S3 storage. To follow up, here is a little guide I wrote on using Duplicity in the everyday work environment Overview Duplicity is a backup tool that will create compressed and encrypted (uses gnupg) backup archives. It can use a variety of protocols as the target (file, ftp, webdav, imap, ssh/scp, rsync, hsi, s3 and hsi). Read More...

cheap two factor authentication with google

published on
I can be a glutton for punishment for a nearly trivial amount of gain. So lets bring on the two-factor authentication for my personal FreeBSD server. I’ve been using Google’s 2-step verification since Jenny told me about it, along with my android powered phone. What is nice about Google’s Authenticator app is its availability for multiple smartphone platforms: Android version 1.5 or later BlackBerry OS 4.2 - 4.7 iPhone iOS 4 or later How it works is pretty simple. Read More...

more fun with ffmpeg

published on
To follow up on two previous posts, the Buckethead concert and making your own YouTube bootlegs, I’m going to add just a little more. As I mentioned, the concert was great. If I were to pick out one part, it would have been this lengthy untitled (as far as I know) freestyle jam he did with a simple looping beat. I cannot think of anything worse than falling in love with a song at a concert, and not knowing the name of it. Read More...

couchdb and ruby on freebsd

published on
I’ve been using Puppet at work for the handful of FreeBSD and, recently, Ubuntu desktops. Aside from some very simple system configuration management (I’ve not yet dived too deep into puppet. I mostly use it for configuring system authentication and ensuring a particular computer security baseline), I though it would have been great to store the client’s “facts” into a accessable database. Sometime last year, Puppet added the ability to store facts into a Couch Database: http://www. Read More...

nlit 2011 vail colorado

published on
Every year, for the last 12 years or so, the National Laboratories have held a technical conference for IT. A different lab hosts it each year, and the National Renewable Energy Lab in Colorado was nice enough to volunteer. I should point out, this conference is not in any way funded by the Department of Energy. It is sponsored by Vendors. I have never been to NLIT before, but this year I felt I had a significant contribution to make and it was worth the effort of submitting an abstract and putting together a nice presentation. Read More...

test drive the cloud drive

published on
todo Amazon recently announced their Cloud Drive, and I feel it is interesting for the following reasons: The ability to play the same music from my phone, my computer at work, my computer at home is pretty cool Everyone gets 5GB of space for free, and you can pay for more When you purchase (DRM free) mp3’s from Amazon, it does not count against your Cloud Drive storage The player is web based, and the tools to download purchased music are usable on Windows, OS X and Linux So, I uploaded some music, bought two songs from the comfort of my phone to play around with it. Read More...

chabot space and science center

published on
A chilly and misty space center We did a night hike, and spent the night at the Chabot Space Center in Oakland the other week. The weather did not permit us to use the large telescopes they have there, but we did get to see the perigee-syzygy of the Moon. It was quite large and nice looking, but too bad the camera does not duplicate the same optical illusion our brains produce by making it look REALLY big: A nice full moon The hike was fun, even when our guide quietly announced “I think we miss our turn…” Read More...

amnesia the dark descent

published on
Amnesia is a great horror game, and one of the most immersive games I have played. It has a dark HP Lovecraft inspired story that you, the protagonist Daniel discovery throughout the game through flashbacks and diary notes. The game starts off with you, Daniel, waking up in a Prussian castle with a note along the lines of, “Hello me, I took something to make me forget the terrible things that have happened. Read More...

meetbsd 2010

published on
I love the BSD community. How cool is it to have developers and end users from the various BSD projects in one location, there to talk about the various cool projects and technical challenges that face us now. Cool I tell you! Chris, Corrigan and I went to Google in 2008 for MeetBSD. We had a great time, so when I saw the announcement earlier this year for MeetBSD 2010, I rallied the troops. Read More...

one brand new another squeaky clean

published on
I’ve had a good run with my Zune MP3 player, excellent in fact, and I would highly recommend that player to anyone who was looking for an iTunes alternative. Truthfully, I’d recommend one no matter what, but it is impossible to beat the ubiquitous nature of iTunes and the iPod, like it or not. I’ve had mine for over 2 years, and I only one had a problem where it locked up. Read More...

freebsd and multipath

published on
I didn’t find any blog posts of discussions on FreeBSD and multipath (for storage) that wasn’t a man page. That means it is up to me to write about it :) Hardware CPU Machine class: amd64 CPU Model: Intel(R) Xeon(R) CPU E5530 @ 2.40GHz No. of Cores: 16 Memory Total real memory available: 65511 MB Logically used memory: 3945 MB Logically available memory: 61565 MB Storage The storage is a large ~90TB Enterprise class Fibre Channel array, a Data Direct Networks S2A9900. Read More...

stuff and things

published on
I don’t have a central theme with this post, but I wanted to at least do something (it has been a while). Packet Filter Based on Chris’s “Falling in love with pf(4)” google status, I decided to take the plunge and move off of ipfw(4) to pf(4). I’m not at the point where I could write my one filter, however, I do feel I at least understand what is happening here. Read More...

gibson sg work

published on
Last month when I helped my Friend Glenn,who now works at An Olde Town Tattoo Parlour in Monterey, move he was cool enough to lighten his load by giving me a very cool cream and black Gibson SG. The original heavy metal guitar made popular by Toni Iommi and Angus Young. The guitar’s neck was broken at the body, and thankfully someone who knew what they were doing successfully glued it back on. Read More...

lagg freebsd

published on
I recently configured a NFS/Samba server with FreeBSD’s Link Aggregation protocol. Here is how I set it up. FreeBSD Configuration /boot/loader.conf I recommend adding the if_lagg_load=YES and kern.hz=“2000” to /boot/loader.conf. The OS will automatically load the lagg kernel module when your network configuration loads, I prefer to explicitly set it to load. ispfw_load="YES" kern.hz="2000" aio_load="YES" hw.igb.rxd=4096 hw.igb.txd=4096 if_lagg_load="YES" Since I am using the igb ethernet device (Intel 82575 and 82576 chipsets), I also set the max number of send and receive descriptors from the default from 256 to the maximum 4096. Read More...

freebsd 8 0 a great nas server

published on
I need to share this. When I google for “Samba performance”, I never see real numbers, real configuration files, or real hardware environments. All I read are anecdotal recollections, and that is not good enough. I like numbers, and I’ll let the numbers speak for themselves: > netstat -I em0 -w 1 input (em0) output packets errs bytes packets errs bytes colls 90166 0 98762637 95363 0 5332847 0 18131 0 24713156 20042 0 1123684 0 4 0 310 1 0 178 0 8 0 518 1 0 178 0 10153 0 10952920 10696 0 598129 0 92990 0 102837002 98476 0 5514994 0 92025 0 102680574 97277 0 5439496 0 92080 0 101799874 97403 0 5448637 0 75348 0 90861608 80972 0 4537737 0 90895 0 100323946 95781 0 5360948 0 89313 0 97371154 94364 0 5278618 0 81363 0 89229738 85861 0 4803589 0 2 0 126 3 0 286 0 I was so shocked that I had to use gstat and zpool iostat to verify the information: Read More...

why you should use disk labels

published on
I recently had a little problem with a new FreeBSD install, and it is one of those times were I sort of appreciate how FreeBSD assigns device handles, yet at the same time hate it :) The setup is this: The OS was installed on a mirrored hardware raid device (using the mpt(4) driver), and then I had a large RAID6 array attached via a FC controller (using the isp(4) driver). Read More...

64bit nvidia driver for freebsd

published on
I’ve always had a vested interest in the entire nvidia display driver for FreeBSD project, and I’ve pretty attached to the project. So much, that back in 2001 I started a little petition, got enough attention (and more importantly, a large list of people who signed my petition), and ever since 2002 FreeBSD users have been able to use high quality nvidia drivers. It wasn’t all me, whoever ran nvidia.netexplorer.org asked me to combine efforts, and I gave them my list, and they continued to market it and work with some folks at nvidia. Read More...

freebsd 8 0 is available

published on
So, it looks like FreeBSD 8.0 has been pre-released; the official date is going to be 11⁄25, as noted in src/UPDATING: Updating Information for FreeBSD current users This file is maintained and copyrighted by M. Warner Losh . See end of file for further details. For commonly done items, please see the COMMON ITEMS: section later in the file. Items affecting the ports and packages system can be found in /usr/ports/UPDATING. Read More...

the coffee experiment

published on
Step 1: The Question Background: I was watching a new episode of Good Eats (Season 13, Episode 7) and Alton Brown was going over a good coffee recipe he called “Man Coffee”. It had a decent ratio of grounds to water, a coarse grind, and he employed the use of a French Press. Then, to my surprise, he added a bit of Kosher Salt. So, being the unapologetic AB fan that I am, I decided to see how the addition of salt would taste. Read More...

windows 7

published on
I’ve been a “PC” user since I was in grade school. Well, technically, the first computer I owned was a hand-me-down Atari ST that had a broken printer and a flight simulator. That almost makes me a “Mac” user since it was based on the same hardware (Motorola 68K cpu). All of my (3) friends had computers, all PC’s running DOS, and their primary motivation was to play all of the cool adventure games from Sierra Entertainment. Read More...

puppetcamp09

published on
This was a very cool conference. I picked up a lot of useful information on both the open source tool, Puppet, and some ideas on infrastructure. What also made this conference unique, is how honest the Puppet team and community were about the projects strength and weaknesses. Those that have deployed Puppet on a larger scale (MessageOne and Google) seemed to go through the same iterations in attempting to scale out their Puppetmaster’s. Read More...

pc bsd 7 1 1

published on
PC-BSD KDE Desktop PC-BSD is a nice mesh between FreeBSD and a ready to use Desktop (which uses about 6GB of disk space). It is based on FreeBSD 7.2, so it has all the cool features of the latest release. Best of all, without ANY additional configuration, I was able to: Use the official FreeBSD nVidia driver for hardware acceleration Watch clips on YouTube (with flashplayer) Play back all sorts of media types like mp3’s, divx, mpegs,wmv,qt… Read More...

zfs updated in freebsd 7 2

published on
FreeBSD 7.x has been using version 6 of ZFS, and originally only 8.0 was going to have the newly updated ZFS version: 13. Last week the core team MFC’d (Merge From Current) the ZFS updates to 7.2, so I cvsup’d and re-build my server’s kernel and world ( with a simple “make buildworld && make buildkernel && make installworld && make installkernel ), rebooted, and now I have the latest ZFS version running: Read More...

while my guitar gently weeps

published on
Over the past year I’ve had a slight resurgence in playing the guitar. Owen likes music, especially the guitar, and I’ve threatened a few times to dust off the electric guitar and really make some noise. After listening to a substantial amount of Pink Floyd, I got the urge (after 10 years of not playing) to pull everything down and get it all set up. I then hit a very unfortunate snag with my old Jackson JS20, the nut had chipped. Read More...

using amazon s3 for backups

published on
I don’t have a backup system for home (which is where this site, and others are located), and I have generally relied on duplicating enough of my important stuff between friends and other computers. That, and I have a RAID5 setup for my large storage, and then home directories and website stuff is on a RAID1 ZFS volume. This doesn’t prevent accidental “oh-no”s, but it does protect me from some hardware failures. Read More...

samba 3028a vs 333 on freebsd 71

published on
!!! UPDATE on 12/29/2009!!! Since this blog post seems to get a good amount or hits from google, if you are reading this, please see my updated post: http://www.mywushublog.com/2009/12/freebsd-8-0-a-great-nas-server/ which has some additional information about FreeBSD 8.0 EOF Lately at work, I’ve been involved with a very large file system that is being export from Solaris 10/ZFS to windows and OS X users via Samba. Even with a very large Sun server (T5220) a lot of users are complaining about the slow performance of the system. Read More...

harman kardon avr 254

published on
todo HK AVR 254 It is no secret that I love watching movies in the comfort of my own living room. I get to eat what I want, drink and not have to worry about driving, and most of the time I’m by myself so I don’t have to worry about shushing people. What can I say, its a great way for me to recharge and get lost for a few hours. Read More...

setting up my own openid server

published on
I’ve configured this blog to use my OpenID accounts. I have two (which totally goes against the single identity mindset of OpenID :) ) http://mcarlson.myopenid.com/ http://openid.evil-genius-network.com/mike The second one I just stood up today. I’m always concerned with who has my information, and if I can, I try to keep it all within the realm of my control. Also, the evil genius domain has absolutely no purpose besides a testing ground that I have no problems destroying :) Read More...

wordpress 27

published on
So, I updated this blog to the latest Wordpress release, 2.7. The interface is all fancy and it didn’t break my current theme so I’m happy. Aside from that, nothing else has been going on. Work has been fun, home has been busy, and I’m still showing up once a week to Wushu West. I’ll have to quickly summarize the movies I’ve seen soon, and since there hasn’t been one that really stood out I haven’t felt compelled to share any. Read More...

meetbsd kip macy

published on
On my last post, someone commented with a full article regarding Kip Macy’s recent legal trouble as a landlord. I don’t know Kip or condone what he and his girlfriend allegedly did, and when it comes to his contributions to FreeBSD it saddens me a little. For a community project like FreeBSD, one persons reputation can reflect on the entire project. However, whats worse is that the commenter,** ‘Mr Scott B’**, feel’s its his role to bring this to the light. Read More...

meetbsd day 2

published on
Thankfully I didn’t have to get up at the crack of dawn (7:30!) because it was scheduled to start at 11am. I was a little rushed for time since I had to drop Caralyne off at my parents in Knightsen (so I back-tracked a bit), and I left at 9am, so I barely made it in time for the ZFS talk. A Closer Look at the ZFS File System by Pawel Jukab Dawidek I’ve heard a great talk on ZFS from Bill Moore, one of the primary developers from Sun, but this talk was really cool because it didn’t just say what ZFS is from Sun’s marketting department’s point of view, but the technical details its mail peices and how it integrates in FreeBSD. Read More...

meetbsd day 1

published on
I spent the weekend at Google for MeetBSD in celebration of FreeBSD’s 15 birthday. I drove 70.2x4 miles, on a weekend, and for Saturday I got up slightly earlier than I would have for work. All worth it, this was the coolest mini-conference I’ve been to. Of course, the last conference I went to was BSDCon in 2003, and that was nice as well. The point is, I don’t get out all that much when it comes to conferences, I just make an exception for my favorite UNIX OS, FreeBSD. Read More...

freebsd 70 zfs and iscsi

published on
A friend of mine got my the coolest birthday present I think I’ve ever recieved, and that was a bunch of new and super kick ass hardware. This will soon replace my current server, which is in such bad shape it cannot compile java code, or perl from source. Before I replace it, I wanted to play around with ZFS that comes with FreeBSD 7.0. Here is a quick rundown of it all: Read More...