Blog Fiasco

March 30, 2014

Parsing SGE’s qacct dates

Filed under: HPC/HTC,mac — Tags: , , , , — bcotton @ 9:19 pm

Recently I was trying to reconstruct a customer’s SGE job queue to understand why our cluster autoscaling wasn’t working quite right. The best way I found was to dump the output of qacct and grep for {qsub,start,end}_time. Several things made this unpleasant. First, the output is not de-duplicated on job id. Jobs that span multiple hosts get listed multiple times. Another thing is that the dates are in a nearly-but-not-quite “normal” format. For example: “Tue Mar 18 13:00:08 2014″.

What can you do with that? Not a whole lot. It’s not a format that spreadsheets will readily treat as a date, so if you want to do spreadsheety things, you’re forced to either manually enter them or write a shell function to do it for you:

function qacct2excel { echo "=`date -f '%a %b %d %T %Y' -j \"$1\"  +%s`/(60*60*24)+\"1/1/1970\"";

The above works on OS X because it uses a non-GNU date command. On Linux, you’ll need a different set of arguments, which I haven’t bothered to figure out. It’s still not awesome, but it’s slightly less tedious this way. At some point, I might write a parser that does what I want qacct to do, instead of what it does.

It’s entirely possible that there’s a better way to do this. The man page didn’t seem to have any helpful suggestions, though. I hate to say “SGE sucks” because I know very little about it. What I do know is that it’s hard to find good material for learning about SGE. At least HTCondor has thorough documentation and tutorials from HTCondor Week posted online. Perhaps one of these days I’ll learn more about SGE so I can determine whether it sucks or not.

January 9, 2014

Online learning: Codecademy

Filed under: Linux,mac,The Internet — Tags: , , , , , , — bcotton @ 9:05 pm

Last week, faced with a bit of a lull at work and a coming need to do some Python development, I decided to work through the Python lessons on Codecademy. Codecademy is a website that provides free instruction on a variety of programming languages by means of small interactive example exercises.

I had been intending to learn Python for several years. In the past few weeks, I’ve picked up bits and pieces by reading and bugfixing a project at work, but it was hardly enough to claim knowledge of the language.

Much like the “… for Dummies” books, the lessons were humorously written, simple, and practical. Unlike a book, the interactive nature provides immediate feedback and a platform for experimentation. The built-in Q&A forum allows learners to help each other. This was particularly helpful on a few of the exercises where the system itself was buggy.

The content suffered from the issue that plagues any introductory instruction: finding the right balance between too easy and too hard. Many of the exercises were obvious from previous experience. By and large, the content was well-paced and at a reasonable level. The big disappointment for me was the absence of explanation and best practices. I often found myself wondering if the way I solved the problem was the right way.

Still, I was able to apply my newly acquired knowledge right away. I now know enough to be able to understand discussion of best practices and I’ll be able to hone my skills through practices. That makes it worth the time I invested in it. Later on, I’ll work my way through the Ruby (to better work with our Chef cookbooks) and PHP (to do more with dynamic content on this site) modules.

January 4, 2012

CNET considered harmful

Filed under: Linux,mac,Musings,The Internet — Tags: , , , — bcotton @ 11:05 pm

In my younger days, I made great use of CNET’s download.com website. It was an excellent tool for finding legal software. Apparently, it has also become an excellent tool for finding malware. An article posted to insecure.org describes how CNET has begun wrapping packages with an installer that bundles unwanted, potentially malicious software with the desired package.

This is terrible, and not just for the obvious reasons. It’s bad for the free software community because it makes us look untrustworthy. There’s a perception among some people (especially in the business world) that software can only be free if it’s no good. I suppose that’s one reason some in the community use “libre” to emphasize the free-as-in-freedom aspect. (Of course, not all free-as-in-beer software is free-as-in-freedom. That’s another reason the distinction can be important.)

When this conveniently-bundled malware causes problems for users, it’s not CNET who gets the blame. Users will unfairly blame the package developer, even though the developer had nothing to do with it. For well-established and well-respected packages like nmap, this reputation damage may not be that important. For a new project just getting started — or for the idea of free software in general — this can be devastating.

February 8, 2011

My thoughts on the Mac App Store

Filed under: mac — Tags: , — bcotton @ 7:44 am

This post proves that this is not a newsy blog.

A few weeks ago, I upgraded my MacBook Pro to Mac OS 10.6.6. With this upgrade, came AppStore.app, the desktop equivalent to the App Store that’s been a large part of the success of iOS. My first impression was “this looks like Novia’s Ovi Store” — it shows a lot of applications and very little information. Looking around, it seems pretty easy to use, but I can’t see myself ever using it.

After years of installing software via `yum install $package`. I got some flak on Twitter for saying this, but the flak was crap. First, I wouldn’t expect anyone to read the man pages for a GUI app on any platform. That’s what the built-in documentation is for (and if it doesn’t exist, that’s a serious bug in the program). Secondly, I wasn’t even talking about the interface. It’s more the idea of paying for the software. Not out of greed, but out of the philosophical feelings about FLOSS.

That having been said, I think the App Store is pretty great overall. My big complaint about Mac OS X is the lack of a package management system. The ability to easily keep packages up to date is a serious strength of Linux distributions, and things like MacPorts and Fink don’t really cut it for casual users. I hope that Apple does the un-Apple thing and makes it more accessible to developers. In the meantime, it’s a great and overdue addition.

July 16, 2010

My TTYtter configuration

Filed under: Linux,mac,The Internet — Tags: , , — bcotton @ 8:07 am

It’s been many months since I found out about TTYtter, a command line Twitter client written in Perl.  Though some users might bemoan the lack of a snazzy graphical interface, it is that very lack which appeals to me.  TTYtter places only a very tiny load on system resources, which means my Twitter addiction won’t get in the way of running VMs to test various configurations and procedures.  Being command-line based, I can run it in a screen session which means that I can resume my Twittering from wherever I happen to be and not have to re-configure my client.

I don’t claim to be a TTYtter expert, but I thought I’d share my own configuration for other newbs.  TTYtter looks in $HOME/.ttytterrc by default, and here’s my default configuration:

#Check to see if I'm running the current version
vcheck=1
# What hash tags do I care about?
track='#Purdue #OSMacTalk #MarioMarathon'
# Colors, etc are good!
ansi=1
# I'm dumb. Prompt me before a tweet posts
verify=1
# Use some readline magic
readline=1
# Check for mentions from people I don't follow
mentions=1

Of course, there are certain times that the default configuration isn’t what I want.  When I was reading tweets in rapid-fire succession during the Mario Marathon, I didn’t want non-Mario tweets to get in the way, so I used a separate configuration file:

# Don't log in and burn up my rate limit
anonymous=1
# Find tweets related to the marathon
track=#MarioMarathon "Mario Marathon"
# Don't show my normal timeline
notimeline=1
# Colors, etc are awesome!
ansi=1
# Only update when I say so. This keeps the tweet I'm in the middle of reading
#      from being scrolled right off my screen
synch

There are a lot of other ways that TTYtter can be used, and I’m sure @doctorlinguist will tell me all of the ways I’m doing things wrong, but if you’re in the market for a new, multi-platform Twitter client, you should give this one a try.

March 3, 2010

Using Mac’s nvram(8) command

Filed under: mac — Tags: , — bcotton @ 9:26 am

I recently came across the nvram(8) command included in OS X.  nvram is used to manipulate the settings of non-volatile RAM, which persists after reboots and power off.  From what I’ve seen, there are about 50 variables that are meaningful to the system, but I haven’t found a comprehensive list so far.  So what is this command used for?  That’s a good question.

One thing you can do is set arbitrary asset tags.  If your organization uses a central asset-tagging system, you can write the asset tag to NVRAM.  You can also set contact information like your name and e-mail address. Of course, none of these options are a guarantee you’ll recover a lost or stolen system. Assuming someone even thinks to look at nvram, the variables could be changed or deleted, or the whole NVRAM could just be wiped.

I asked Twitter if anyone had uses for nvram(8) and no one seemed to.  I’ll leave it open to my readers to suggest uses for this command.

December 28, 2009

Filename extensions can cause problems

Filed under: Linux,mac,Musings — Tags: , , — bcotton @ 10:08 am

Most people don’t really give much thought to the idea of file extensions, although they’re nearly universally in the minds of modern computer users.  Users have come to understand that .pdf means a file is in the Portable Document Format, or that .ppt is a Microsoft PowerPoint file.  DOS users recall that files ending in .exe, .com, or .bat are executable. For those unknown extensions, there’s the very helpful filext.com website.  There’s no doubt that filename extensions can provide very helpful information, but here’s the issue: not all platforms care about them.  That’s not a problem in all cases, but there are times when it makes life miserable.

Filename extensions can be just another part of the filename, or they can be entirely separate namespace.  DOS first introduced the idea of extensions to the general public.  In those days, the file had a name of up to eight characters, and an extension of up to three.  This “8.3″  convention persisted into Windows, and is still commonly seen on Windows system files, even though it is no longer necessary.  Unix-based systems, such as Mac OS X and Linux, have no feelings about extensions — they’re certainly not required, but some applications make use of them.  The dominance of Windows in the desktop market has encouraged application writers to really care about extensions, and it does help in trying to find the right type of files.

Here’s where it becomes problematic.  Because some systems don’t care about extensions, it’s easy to not have extensions on your filename.  Then, when you go to a system that does care, things don’t work as you expected.  Here’s a fine example: my wife needed to have a few pictures printed, so she loaded them onto an SD card and took them to the store. When she got there, the photo system would not find any of the pictures.  As it turns out, she had saved them without the .jpg extension, so while they were valid JPEG files, the system didn’t try to load them.

Now, most photo software, cameras, etc. will add the extension out of tradition (and because that’s what people expect). However, a manual renaming of the files after the fact could result in absent extensions.  So what is the solution?  Well, we’ll never get all platforms to come to agreement on what filename extensions are, and how they should be defined and treated.  The only answer, then, is that applications should be written to not focus on extensions, but on the contents of the file.  If applications used methods similar to the Unix file command to determine file type, then such problems could be avoided.

November 23, 2009

Using bookmark synchronization on Google Chrome for Linux and Mac

Filed under: Linux,mac,The Internet — Tags: , , , , , , , , , , , — bcotton @ 8:18 am

For a long time, I blamed the sluggish performance of the web browser on my Linux machine at home on the ancientness of the hardware.  However, when my much nicer Linux machine at work showed the same problem, I began to think maybe it was just Firefox.  I’ve been an avid Firefox user for many years, but my loyalty wavers when my browser can’t keep up with my keyboard.  Based on the advice of strangers on the Internet, I decided to give Google’s Chrome browser a try.

Chrome is still a maturing browser, but it is fast and capable.  There’s only one real drawback: bookmark synchronization.  With Firefox, I had been using Xmarks to synchronize my bookmarks, but that’s not currently available for Chrome.  In the “Early Access” builds of the Linux and Mac versions of Chrome, the bookmark sync that the Windows version has is available.  This syncs the bookmarks to your Google Docs account, which makes it rather handy.  However, synchronization is not enabled by default.  To enable it, you have to pass the –enable-sync option at launch time, which is easier said than done.  Fortunately, it’s not too terribly difficult.

(more…)

November 17, 2009

Setting up a new Mac

As part of my new job, I got a shiny new 13″ MacBook Pro.  Even though I’m quite a Linux fanboy, I really enjoy the quality of the hardware and OS X. However, it isn’t perfect.  There are a lot of applications that I like to have available.  Since I have nothing better to talk about, I figured I’d list them here:

  • Adium — one of the best instant messenger clients I’ve ever used.  It has support for just about every major IM protocol except…
  • Skype — I don’t really use it for IM, but it’s great for audio and video calls.
  • Firefox — I prefer it to the Safari browser that ships with OS X.  It happens.  And with that comes…
  • Xmarks — a browser plug-in that syncs bookmarks.  It comes in very handy when you use multiple computers.  So does…
  • Dropbox — allows you to synchronize arbitrary files between multiple computers.  I mostly use it for configuration files (e.g. .bashrc, .screenrc)
  • VirtualBox — sometimes you actually need to use another OS to do some important task (like play Sim City)
  • DOSBox — is good for playing some of the older games that I like
  • Chicken of the VNC — I’ve played with several VNC clients for Mac, and this one is the best.
  • iTerm — hands-down better than the default Terminal.app
  • ZTerm — a program to make serial connections.  I used it a fair bit in my old job, I don’t anticipate needing it much in my new job.
  • Colloquy — an Internet Relay Chat client
  • VLC — a media player that will play just about anything
  • Grand Perspective — a program that shows a graphical representation of disk usage, allowing you to find the files that are chewing up all the space on your hard drive.

November 9, 2009

Accessing Taleo from Mac or Linux

Filed under: Linux,mac,The Internet — Tags: , , , , , — bcotton @ 12:47 pm

Some companies (including my own employer) use a company called Taleo to manage the hiring and recruitment process.  As an applicant, I’ve not been very impressed with it, but that’s neither here nor there.  From the applicant side, you can use just about any browser to fill out the forms and submit your application.  However, if you’re a hiring manager, Taleo expects that you’ll be using the Internet Explorer browser.  If you’re on a Windows machine, that’s probably available to you.  For Mac and Linux users, it’s not an option.

So what can you do?  You can either go find a Windows machine to use, or you can try to run Internet Explorer using Wine.  Neither of those are necessarily that appealing.  Fortunately, there’s a third option, which is to use the (closed-source but free-as-in-beer) Opera browser.  Once you’ve got that installed, it’s a quick process to get Taleo workin* In Preferences, click on the “Advanced” tab and select “Content”

  • Click the “Manage Site Preferences…” button
  • Click the “Add…” button
  • Enter your Taleo site (e.g. “company.taleo.net” or “taleo.com”) in the “Site” field
  • Choose “Open all pop-ups” in the “Pop-ups” drop-down menu
  • Click the “Network tab”
  • Select “Mask as Internet Explorer” in the “Browser identification” drop-down menu (note that “Identify as Internet Explorer” will NOT work)
  • Click “OK”
  • Click “Close”
  • Click “OK”

That’s all it takes.  As a bonus, you now have the very capable, stable, and secure Opera browser installed.

Older Posts »

Powered by WordPress