Slack Pros and Cons

I love Slack, but some of the people where I work do not like it.  To be somewhat fair, I will examine some of the pros and cons.

Pros

  • Designed by software developers to help them do software development.
  • Channels.
  • Easier-to-use interface than IRC.  You don’t have to maintain an IRC server.
  • Integrates with many other systems.  REST API.
  • Mobile, web, and desktop.  Cross-platform.
  • Starred items, searchable conversation history, ad hoc group chats, fast channel switching.
  • Reminders.  It’s amazing how often people say they can’t remember something when they’ve got Slack reminders, Outlook/Google calendars, physical/virtual sticky notes, and more to help them.
  • It’s way better than email for real-time group collaboration.
  • Attachments and code snippits.  Try telling someone how to enter 3 tricky Linux commands over the phone.
  • Easy to drop pictures, emojis, and reactions.  The /giphy command.
  • Easy for people to tap into the collective knowledge of an entire department to get people unblocked while at the same time avoiding the same question being asked over and over.
  • Far more efficient for quick exchanges of information than email.  Avoids all the pretentious boilerplate salutations and signatures.
  • You can start group video calls inside Slack.
  • It’s a lot harder to get sent random spam since only people who are part of your Slack group can contact you.

Cons

  • Designed by software developers to help them do software development.  Business users may find it too feature-rich.
  • A comment on a thread within a channel does not mark that channel as unread.
  • No shortcut key to begin a new thread.
  • You can’t customize shortcut keys at all.  You can work around this with AHK, but again, now you are alienating non-programmers.
  • No abbreviations for / commands.  Again, I have to use AHK.
  • Can be overwhelming/distracting if a lot is happening on a lot of channels.
  • Email may track conversations better than channels/threads in some cases.
  • It’s yet another communication channel to monitor.
  • It’s yet another tool to learn.
  • There’s no way to customize the delays for snoozing reminders.
  • Would be useful to have more ways to tag posts than just starring them.
  • In-app video calls aren’t as good as Skype.
  • Somewhat expensive.  This becomes a problem when an organization says “We already paid for Microslop BarfChat, so use that even though our organization will be half as efficient.”
  • Even Slack (which isn’t hard) is too hard for some people.  In reality, your organization probably needs both an insanely simple chat tool for all users AND Slack for software development and other more technical communication.
Advertisements

Barriers to Bottom-up Communication

I’ve been thinking about organizational communication lately.  My niece graduated with a degree in it this year.  I started a new job this year, and have been analyzing communication within a much larger organization.  Most attention gets given to top-down communication, and even that tends to break down.  Bottom-up communication tends to be in much worse shape.  These are my observations as to why.

ego

If you believe you are more important, intelligent, whatever than the people talking to you, first, you’re wrong, and second, you won’t even acknowledge them.  I’ve seen it go to the point of a CEO ignoring the unanimous input of his entire tech team on technical issues.  You just don’t understand, Jeff–the point isn’t to be right; the point is for me to be right.

apathy

Sometimes people just don’t care.  Maybe it’s tenured faculty or the boss’s son.  Very little will happen to them if they don’t care.  They don’t want to deal with anyone’s problems.  They are checked out.  Ideally, these people would not be employees, let alone leaders, in your organization, but it does happen.  These people’s attitudes breaks the bottom-up chain of communication completely.

dismissiveness

Tends to require heavy doses of ego.  Deciding that an issue is not important just because it is not important to you.  It’s probably less costly to just listen than to force people to make you listen.

intentionally ignoring people

This is a bit different from dismissiveness in that it is more calculated.  If you’ve ever gotten the corporate run around of being transferred between 5 different departments only to have your call dropped, I can tell you it’s not always much better inside a large organization.  I see this more in peer-to-peer communication, especially across departments.  I didn’t see much of this in a smaller organization.  Everyone knew each other, needed each other more, and therefore tended not to cavalierly dispense disrespect.  I used just need where I kept track of things I was waiting for.  Now, I have to keep a list of things I’ve had to completely drop due to the total apathy of others (even after following up multiple times).  People take the stance that since you can’t make them do anything, they should ignore you by default.  It’s an extremely greedly optimization that causes massive damage to the global organization.  The danger of this is that it leads to mirroring of the same behavior.  If that person or their department needs something critical in the future, everyone who got blown off in the past is going to remember that.  The silos go up and become fortified–the distrust and disrespect slowly grow into all out war.

illusions of communication

We have a weekly meeting that anyone can theoretically call.  The misguided idea is that it is more efficient not to have a meeting if there is no need for one.  The reality is that no one calls the meeting (except the boss) because your one issue would have to be significant enough to use a lot of people’s time.  Now the aggregate small issues of everyone get ignored.  Even worse, if anything ever comes up, you’re to blame because you could have called a meeting.  It’s a complete illusion that acts to suppress communication while appearing to be a communication mechanism.

Let’s talk about fake Scrum standups.  If 95% of the meeting is watercooler chat, smoke blowing, and posturing with 5% devoted to blockers that just go into a black hole, all you have is the illusion of communication.

Another illusion of communication I see is car dealers, dentists, etc. with online forms.  They seem to want the illusion of having a web presence.  If you are not going to even respond to the form or just tell that person to call someone, don’t bother putting the damn form up!

actual bandwidth limits

It is possible that someone is getting hit with so many messages that they just can’t keep up with them.  In this case, there is usually one of two problems (1) the person in power is unwilling to give up power, (2) the company is trying to do more than it has the resources to do and just needs to focus.  In the first case, more people need to be hired and power delegated to them to respond to incoming messages.  Usually ego prevents this from happening.  In the 2nd case, ego usually causes a company to overestimate its abilities and choke to death on its own ambition.

completely not monitoring bottom-up channels

Sometimes there’s plenty of bandwidth, but people are just not paying attention.  If you have a Facebook page intended to facilitate communication, but then you just post to it and never check it, it’s just one-way, top-down communication.  Maybe that’s the intent; maybe it isn’t.

I wrote weekly status reports at my last job.  There was a section where you could raise problems.  It’s clear that those status reports were rarely, if ever, read.

no intentional design

I think often almost no thought goes into intentional design of bottom-up communication mechanisms.  Mechanisms are simply in place by inertia and tradition.  Leaders don’t challenge themselves as to whether what they are propping up a broken system or even a complete sham.  If issues are not being raised or simply dismissed arbitrarily by any link in the chain, then the system is broken.  A system the could be used for communication is not the same as a system that actively extracts and propogates communication.

 

 

 

 

 

Scrum Blockers

Scrum is a nice idea, but I’ve never worked in a situation in which it actually worked.  Here are some of the realities that cause Scum to fail.  These are not easy metablockers to remove!

  • Culture of blame turns standups into “who can sound the most productive” posturing meetings because no one wants to be blamed.
  • Blockers raised in standups enter the upper management black hole of apathy.  Indeed, raising issues just gets you blamed for them, so you are trained to shut up and make your own lifeboat (update your resume) while the ship sinks.
  • Team does not use a shared kanban to report status thus wasting time in standup reporting status rather than addressing blockers.
  • Most people on the team have merely heard of Scrum–they have no idea what it actually is.
  • There is no upper management or stakeholder support.  You’ll be lucky to get feedback on your project once every two months let alone once every two weeks.
  • Development is not actually being done by teams.  A group of devs are actually each individually working on completely separate projects.
  • Being transparent is just handing ammunition over to someone else to politically destroy you.
  • Sufficient automated build/test/delivery infrastructure does not exist to even allow a team to iterate quickly.
  • Estimates produced by velocity tracking are simply ignored by delusional hyper-optimistic upper management.  There’s no way the project will take 7 months when they’ve already promised delivery to the customer in 2 without consulting the dev team.
  • One person is expected to be the Scrum Master, Product Owner, and Lead Developer for multiple projects.
  • Upper management habitually bypasses the Product Owner, whimsically dumping new tasks into every sprint and habitually blaming others for the consequences of their micromanging maniacal meddling.
  • The compensation structure of the organization does not reward exceptional performance.
  • No one has full stack development skills.  Developers get caught up in what their title should be relative to other employees.
  • Retrospectives are just witch hunts.
  • Managers want the usual status reports in addition to the status artifacts Scrum already provides.  Neither are actually read nor responded to.
  • Employees in the trenches really don’t care about improving company performance as long as they’re still getting paid.  On the flip side, management doesn’t have the will or authority to fire dead weight and excessively obstinate change resistors.

Software “Releases”

Okay, I’m up on my virtual soapbox regarding software releases today.  Here are some indicators to let you know that you haven’t actually created an actual software release.  If too many of these things are true, you’ve just slopped together some random crap you have no control over.  Releasing a non-release is like mixing matter with anti-matter: you are courting spectacular disaster.  I’ve been personally bit by all these issues, so I know what the explosions feel like.

  • Anything in your “release” is not traceable back to SCM or an artifact repo.  For example, if your sysadmin whipped up a script used in production that never finds its way back into Git or whatever, I call this a “rogue script” because it exists outside of the control of the normal software development process.
  • Your “release” requires manual hand hacking to get working.   No!  Thou shalt not hand hack thy production environment.
  • Your “release” can’t be successfully installed by a non-technical person given the installation instructions.
  • Your “release” overwrites existing configurations/customizations without extremely good reason (laziness and bad architecture don’t count) and ample warning.  Yes, I’m talking to you Windows–you screw up my keyboard configuration every damn time.
  • Your “release” and all its components do not have versions.
  • Your “release” has no release notes and does not document the version of all its components.
  • After installing your “release”, a system administrator cannot easily determine the version of your system and all its components at runtime.  This includes firmware.
  • Your “release” does not have system tests that (among many other things) can verify nobody has hand-hacked any deployed files after installation.  Nope, you can’t trust your own coworkers or customers not to willfully or accidentally do this.
  • Another department within your company can’t deploy your “release” even with your help let alone completely without your help.  And yet somehow you’ve decided this garbage is ready to give to an actual customer.
  • Your global head of software tells you your “release” is not ready to release.  It astonishes me (actually it doesn’t–arrogance has no bounds) that some executives will pay for competent advice only to ignore it.  Next time my dentist tells me I have a cavity, I’m just going to say “No, you’re wrong, I don’t.  Thanks for your opinion.”

Rude User Interfaces

It seems like I’ve been running into rude user interfaces left and right these days.  Because of this, I am going to enumerate some of the characteristics of rude user interfaces (which are also characteristics of rude people) in the hopes that at least I will remember not to commit these crimes against users.  I have been a victim of all these crimes.

making the user repeat themselves needlessly

  • We have some homebrew project tracking software at work.  Any time another field fails validation, it clears your notes field.  Rude.  Sure, things like Lazarus can help with that, but how many times have users without such a plugin had their time wasted by such rude user interfaces?
  • The auto mechanic I currently use has an online appointment system that lets you enter what services you want.  However, every time I show up, they ask me “So, what are we doing for you today?”  They are too lazy to read their screen, so they inconvenience me with restating what I already told them.
  • Every address form that asks for the city and state before the zip code.  Wrong!  You could automatically fill in the city and state for the user from the zip code they provide.

overzealous validation

  • Suppose your web form has a number field.  You decide that you are going to validate each character entered so it can only be a number.  Wrong!  Your user interface is rude.  Some people use text expansion software so they don’t have to remember long numbers.  You should only validate input in its final form.
  • Suppose you have a field that accepts a phone number.  The user enters just numbers, and you inform them that they need to enter (s or -s or other formatting characters.  Wrong!  Your user interface is rude.  You are making the user jump through hoops that you already know how to jump through for them.

making the user switch between mouse and keyboard for a single task

  • You have a password field.  You’ve been nice enough to provide a little eyeball to click to see the password (some UIs are even so rude as to not provide that).  But, you failed to provide a keyboard shortcut to show the password.  Now the user has to switch from keyboard to mouse and back.  Rude!
  • In general, not making your UI keyboard navigable is rude.

assuming everyone uses the same keyboard layout

  • You have some form that accepts numbers only as typed (again, bad idea).  You add insult to injury by validating on the keyboard scan codes rather than the actual character events.  Thus, you force the user to switch keyboard layouts just to fill out your extremely rude number field.

making user select from a zillion unlikely options

  • You know the user is from the US using US English.  But, to be rudely internationalized you present an alphabetical list of all the countries instead of putting USA at the top.  This is the equivalent of answering the question “Can you get me a drink?” with a yes, doing nothing, and then making the person restate the request as “Will you get me a drink now?”  Rude.

calling a field something it is not

  • 7 Days to Die has a field called “IP address” that you can put a hostname into.  This is misleading and therefore rude.

nested scrollbars

  • No.

self-managed windows

  • Toad, you are so very rude in this regard.  Why don’t you use normal windows that behave like other windows.

removing options placed at the top from the main list

  • Okay, you’ve been nice enough to put United States at the top of your country list.  However, your intrepid user is so used to being mistreated that they don’t even notice it and resign themselves to scrolling through the massive country list–only to find it is not there.  They double check, then curse your name.  You’ve done them wrong!  Do not remove the option from the main list just because you provided it at the top for convenience.

overzealous time outs

  • "Welcome to the real world", she said to me
    Condescendingly
    Take a seat
    Take your life
    Plot it out in black and white

    A couple minutes later, “your session has timed out–please log in again.”  And of course, the one minute of your life history you managed to jot down has been erased.  How rude!

making the user remember something you already know

  • In the early days of Google Play, it could not do automatic video playback bookmarks even though Amazon and Netflix had managed to do them since the dawn of time.

overwriting UI configuration on update

  • Windows, you are so rude.  Every time you update, I have to reinstall the Programmer Dvorak keyboard layout and fiddle with settings.  Somehow Linux has managed to incorporate this layout in its built-in options and not override my settings each time it updates.  You make me type my password with the onscreen keyboard and don’t even set focus to the only text field on the screen when I pop it up.  You get a rudeness gold medal for this one.

not learning how to do your job

  • I sometimes ask Cortana for specific options within Windows by exact name, and she can’t find them.  Learn how to do your job, Cortana.  Also, if she was good at her job, I would not feel the need to install Launchy.

giving the user useless crap that was not requested

  • Kindle, I do not want your extra citation crap when I copy something.  I do not want your funky nonprintable characters.  Nor do I want you to wreck the indentation of copied code snippets.  Also, you can’t seem to sync my categories across multiple instances to save your life.  Nor do you sync PDFs I import.  You truly are a bastion or rudeness.
  • Spam and ads fall into this category of rudeness.

TMI

  • I had Box send me a notification that it successfully copied something to the clipboard.  Thank you so much, Box, for that valuable status report.  Perhaps you can elaborate on how much memory was used in that operation, what other processes were running, the time in milliseconds when this occurred and other fascinating, important details of such an amazing accomplishment.

vague and/or cryptic error messages

  • A popup appears saying “An error has occurred.”  Great.  That explains everything.  I know exactly what to do now.

The Search for Search

Where Are You Searchmas?

Where are you Searchmas?
Why can’t you find me?
Why have you gone astray?
Where is the data, you used to bring me?
Why can’t I find naught but hay?

At the top of my Christmas list this year: good search mechanisms.  The ability to find the right thing quickly is a large part of what has made Google and Amazon such valuable companies.  I will talk about some software that falls short of the Searchmas dream.

GitHub

I can’t search across the documentation of multiple private repos in an organization on GitHub.  It’s hard enough to get anyone to document anything in the first place.  Not being able to find it afterwards is sad.  I did find a GreaseMonkey script to do this once, but I think it stopped working.

Windows 10

Sure, most people only have at most one printer, so it’s easy to select that printer from a list.  However, I recently had to find a printer in a list of at least 500.  Windows provided no search option in this case.

Also, if you want to change a file association in Windows, it provides no search mechanism for this.  You have to scroll through a list of hundreds of them.

Windows Explorer has no regular expression based search capability (at least last I checked).  This is a crime against humanity.

I am absolutely sure these are not the only two places Windows 10 has miserably failed to provide search capabilities when needed.  You’d think all the spyware they added would clue them into these issues.

An Online Directory

I recently used a directory service to try to look up my own name.  I entered “Jeff”, but it could not find me because I was in there as “Jeffrey”.  Fail.  A good search needs to know about common aliases and do some level of fuzzy matching.

IntelliJ Search Everywhere

Some aspiring-to-be-helpful IntelliJ tooltip said I could press shift twice to “search everywhere”.  This was great, I thought, because I needed to find some file in an unfamiliar project I had seen once.  I knew there was a particular string in that file.  So, I searched everywhere for it.  Got back some obscure method call in some library, but not the file in my project.  Eventually navigated to the file and found the string.  Even with the file open in IntelliJ, “search everywhere” could not find it!  Now, maybe I don’t understand what that feature does, but if you are going to call something “search everywhere”, it should search everywhere, dammit.

Launchy

While I do love me some Launchy, it drives me nuts that I can’t set exclusion rules for specific directories.  If they ever fix that, I do pledge to finally donate some money to their cause.  Certainly not a whole Bitcoin.  Maybe a tulip or two’s worth.

Conclusion

Decades of software development, talk of AI taking over the planet, and we still struggle with the basics.  Even giant companies with billions of dollars struggle with the basics.

Miniature Rant Against the Entire Global Software Industry

A large part of this is because these giant software companies want to hoard cash robbed from their engineers (hello Apple), collude to bring developer wages down (hello Apple, Google, Intel, Adobe, etc.), try to convince everyone that software development is easy (hello Hour of Code and other such shallow nonsense), and import/export from/to whoever they can (hello India) instead of paying the real talent what they’re worth.  Don’t call people rock stars and then pay them like roadies.  I certainly feel, as I’m sure many do, repeatedly stabbed in the back by the entire software industry after devoting the real, long, hard time necessary to becoming a decent software developer.  The effort is on par with becoming a doctor, but the pay certainly is not.  Though cheated, we get no sympathy from the public because we are “well paid”.  Silicon Valley tech bros sneering at the homeless and building robots to shoo them away does not help things.

Finally

And so, the search for search continues.  Let us demolish all bad searches so that we, as all restaurants in the future, can finally say: Search Mas.

 

 

Bash Tutorial: Getting Help

Getting Help

 Let’s face it: you need help.  A lot of help.  Your mom has known this for quite a while now, but you are just beginning to realize it.  So, let’s explore what you can do about it when she is not around.

 Let Me Google That For You / RTFM

Before you ask your pressing question to the world on Facebook, mailing lists, online forums, Twitter, and the corporate global e-mail address, first pause and do nothing.  Before you get down on your knees and pray for the answer to your pressing question (e.g., “What does the echo command do?”), reconsider.  Type your query into Google first.  You may be astonished to find out that your question has more often than not already been answered.  Others have been confounded by the very same mystery you are stuck on.

 Here are some websites where your question has already been answered:

http://stackoverflow.com/
http://www.tldp.org/
http://www.gnu.org/software/bash/manual/bashref.html
http://www.linfo.org/main_index.html
http://mywiki.wooledge.org/BashGuide

“That’s all well and good, Jeff,” you say, “but I’d have to lift my hand off my keyboard, move it to my mouse, launch a web browser, go to Google, type in a query, and hit enter to do that.  I have enough hassle in my increasingly short life.”  Wow, I’m surprised you’ve read this far given your fierce work ethic and indomitable spirit.  Fear not, if you have Launchy installed (and Firefox as your default browser), you can do it all from the keyboard.  Ah, all the fools and their pathetic touch screens!  Type <Alt + Space>google<Tab>Bash<Enter>.  Now <Tab><Enter>.  Bam!  You are on the Bash Wikipedia page without the hassle of using your mouse.

If you can’t figure out how to install Launchy, TRY GOOGLING IT!  If you can’t figure out how to configure it, RTFM!

 When the Internet Is Broken

If the unthinkable occurs, you can still get help from your trusty computer.  Fortunately, not everything has been assimilated by The Cloud yet.

$ man bash
$ info bash
$ help <built-in>
$ <non-built-in command> –help

When I use <foo>, I don’t mean type <foo> literally.  I mean “replace <foo> with something that is a foo”.  In this case, type something that is a Bash built-in command.  If you just type ‘help’ by itself, you will see a list of Bash’s built-in commands.

A built-in command is a command that Bash handles by itself.  Non-built-in commands are compiled binary executables or other scripts that exist as separate files on your computer.  Often they will support a –help command-line option.  Types of commands, command-line options, command-line arguments and a bunch of other related topics will be discussed in more detail later.

 Helping Yourself

Well, now you’ve learned a bunch of useful stuff, and what did you do with it?  Nothing.  You did not write it down.  You will not remember it.  Nice work, Einstein!  As Gandalf would say, “Fool of a Took.  Throw yourself in next time, and rid us of your stupidity. “  May I suggest a different approach: write down your hard-won knowledge.  When God talked to Moses, was that it?  Did he wander off and think, “Moses is a sharp guy–he can remember 10 things.”  No!  He wrote that stuff down on a couple stone tablets (the very ancient equivalent of CD-Rs).  You also have the ability to write down important things.  Suppose you learn something interesting about the echo command.  Maybe you could write that information in a file like ~/Dropbox/Notes/Linux/Commands/echo.txt.  Maybe that file is in Dropbox.  Maybe that file is indexed by Launchy so you can type <Alt + Space>echo.txt<Enter> to get to it really, really fast.  You may end up with a focused, quickly accessible database of knowledge pertaining to the things most relevant in your life.  Perhaps it is even under revision control on a machine that is backed up regularly.  Then, it really is like stone.

Another good way to help yourself is to write what I call ‘hello projects’.  These are simple one-file projects focused on the basics of a topic in some language.   Let’s say in the near future, you are learning about the basics of for loops in Bash.  You suddenly have the brilliant idea to make a hello project: ~/Dropbox/Projects/Hello/Bash/hello_for.sh.  Now, whenever you forget about the basics of for loops in Bash, you can look at a very focused simple piece of actual working code.

Over time, instead of asking people annoying questions, you will waste less of their time by answering them yourself.  I salute you for your contribution to society.

 If All Else Fails

If you have completely failed in your quest for knowledge, humble yourself before your fellow humans and ask a question.  While you will suffer humiliation for a second, you will not remain in ignorance for an eternity.  Asking random people at the mall will not work.  People over 60 may give you a blank look.  Small children (and many grownups) may start crying.  Try instead a tech-oriented co-worker, the mailing list of a local Linux User’s Group, or a relevant online form.  If you are still not getting results, try one of these: http://www.amazon.com/Mattel-30188-Magic-8-Ball/dp/B00001ZWV7/.  Or just ask your mom (at least if you are Dilbert).

Python 3: Keywords

I’m learning Python 3 at the moment.  I’ve done some Python 2 before.  Working through Learn Python 3 the Hard Way is interesting.  Lots of making little text adventures.  Interesting to me because I played a lot of them in their short golden era.

Anyway, the author has you periodically enumerate all the interesting symbols you’ve seen in the language and briefly state what you think they mean.  I’m going to do that just for Python 3 keywords here.

and
Logical and with shortcut evaluation.
True and False == False

as
Rename symbols when imported; also used with “with as”.
from sys import argv as clas # Command-line args.
with X as Y: pass

assert
Assert that a boolean expression is true.
assert False, "Error!"

break
Exit from innermost loop.

while True:
    if 1 == 1: break

class
Define a class.
class SoftwareDeveloper(Superhuman): pass

continue
Start innermost loop again (skip any remaining instructions in current iteration).

while True:
    if 1 == 1: continue

def
Define a function.
def myfunction(arg): pass

del
Delete an item from a dictionary.
del d['property']

elif
Start an else if block.
if : pass
elif : pass

else
Start an else block.
if : pass
else: pass

except
Catch an exception.
try: pass
except ValueError, e: print(e)

False
Boolean false literal.
False

finally
Execute this block even if exceptions were raised.

try: pass
except: pass
finally: pass

for
Loop over an iterable.
for element in list: pass

from
Import specific symbols from a module into the current namespace.
from sys import argv

global
Declare that the variable you mention is a global, not a local.
global some_otherwise_local_variable

if
Start an if block.
if True: pass

import
Import a module as a whole.
import sys

in
Check if an element is contained in something else or form for loops.
if choice in "Yy": pass
for element in list: pass

is
Tests equality somewhat like ==. Is this like Java equals()?
1 is 1 == True

lambda
Create a short anonymous function.

squared = lambda y: y ** y
value = squared(4)

not
Logical not.
if not 1 == 1: pass

or
Logical or.
True or False == True

None
Literal value for NoneType.
o = None

nonlocal
This is not local, but it’s not quite global either.
nonlocal var_in_some_enclosing_nonglobal_nonlocal_scope

pass
Do nothing (used to create empty block).
def noop(): pass

print
Okay, this is not a keyword. Builtin to print string and trailing newline.
print("Hello, Python 3!")

raise
Raise an exception.
raise ValueError("No")

return
Return a value (can be a tuple) from a function.
return result

True
Boolean true literal.
True

try
Start an exception handler block.
try: pass

while
Start a while loop.
while True: pass

with
Do a block with an expression as a variable.
with X as Y: pass

yield
Pause and return to caller. Used in generators.
def x(): yield y; x().next()

Bash Tutorial: Basics

Foreword

I wrote a few Bash Tutorial articles back in 2014.  I am now reposting them here.

It is very hard to write a very basic tutorial on anything having to do with software development.  Anyone that has developed software for years is steeped in a massive amount of background information that is impossible to completely unassume.  Now that I’ve washed my hands of all responsibility for failing to teach you anything (and presumptuously done so using the nonword “unassume” which I only assumed existed when I should have been more unassuming), I shall proceed.

What is Bash?

Bash is the “Bourne Again Shell”.  It is the successor to the “Bourne Shell”.  AFAIK, it has nothing to do with The Bourne Identity series of movies.  Nor is it a religious movement.

So what is a shell?  A shell is an enclosure that protects the thing inside from the stuff outside.  In this case, you are the stuff outside trying to destroy the inner workings of your computer.  By this definition, shells have not been very successful.  You’ll end up destroying your computer despite its efforts to prevent this.  More specifically, a shell is a program that lets you interactively issue commands to your computer.  Without a shell, you’d have to write lower-level programs in languages like C or C++ that make calls to system libraries whenever you wanted to do anything interesting on the computer.  That’s no fun, so we software developers have spend thousands upon thousands of man hours building layers upon layers of software to make it easier for you to talk to your computer.  And all you end up doing is posting a picture of your lunch on Facebook while watching a funny cat video on YouTube.  You’re welcome?  Anyway, a typical computer system has these layers:

hardware
dust
NSA-infected or Chinese-infected firmware
kernel and drivers
system libraries
Facebook, spyware, adware, and anything from Norton
shell
screen
the sticky notes comprising your enterprise project management system
your password, ‘password123’, written on the back of said sticky note
keyboard
dusty crumbs
a cat napping on the keyboard
mouse
you (at least the 15 minutes of the day when not on break, at lunch, or in a meeting)

You can consider a graphical desktop environment as a visual shell.  Bash is a text-based shell, that is, a command-line shell.  You type in text and get text back as output, along with other side effects like deleting all your files.

Why Should I Care About Bash?

Unless you are or are trying to be a programmer, sysadmin, or something similarly odd, you probably won’t care.  You probably stopped reading once you hit the words “computer” or “definition” and started reading my PowerPoint for Winners series.  You believe all necessary computing problems have been solved by Amazon Prime Pantry, the popcorn button on your microwave, and Netflix.

If not, a shell lets you do the same kinds of things you can do in a normal desktop environment: create files, move files, delete files, search for files, manage directories, create links/shortcuts, launch programs, terminate programs, etc.  For certain activities, a command-line shell like Bash tends to be (far) more efficient and powerful than a visual environment.  Also, Bash is the standard shell on Linux and is available on Windows machines via Cygwin.  Since OS X is Unix-based, Bash is available there too.  Basically, whatever environment you are working in, you’ll likely have the phenomenal cosmic power of Bash available.  So, why not use it?

Another advantage of a textual shell, is that it is far more scriptable than a visual shell.  Sure, you can use AutoHotkey (AHK) to do cool stuff on the desktop, but it’s more of a pain.  The same commands that you type into the shell interactively can be put directly in a file and used as a script.  This lets you automate tedious things that would take you hours to do by hand.  Instead of wasting your time on all that tedious stuff, you’ll get to waste your time on trying to figure out why your scripts aren’t working!  I’ll give you a hint: you either have whitespace where it shouldn’t be or don’t have it where it should be.  But seriously, hopefully your automation efforts will produce a net gain.  I have seen people’s entire jobs replaced by relatively simple scripts.  You want to be the person writing the script, not getting replaced by it.  The robots are coming, my friends.  In any case, it will be more fun, and more useful to put “Bash scripting” on your resume than “google”, “double click”, and “drag and drop” in the skills section.  Some people have not sufficiently acquired the “google” skill yet though.  For sheer resume points, knowing Python is more valuable.  However, shell like Bash still shine in their interactive/tactical capabilities more than Python (which is a pure scripting language not intended to be used as an interactive shell).  You can script in Bash, but it is more of an awkward side note.  A very direct, portable, widely-used awkward side note, however.  Really, you want to know all three: Bash, Python, AHK.  There’s very little tactical programming that cannot be accomplished with those three.

How Do I Get Bash?

This tutorial is written from a Cygwin/Windows 8 perspective.  Cygwin is a Unix-like environment that runs on Windows.  If you install Cygwin, you will have Bash.  It will be the default shell that runs when you run Cygwin.  If you are on a Linux machine, you likely just have to open a terminal program and you’ll be running an interactive Bash shell.

 So, download and install Cygwin.  I do an everything-and-the-kitchen-sink installation.  This will likely take hours to complete, so be patient.  You could do a full Ubuntu install on a VM faster.

http://cygwin.com/install.html

 You should now have a shortcut on your Windows desktop called “Cygwin64 Terminal” or something similar.  Launch that, and you should now be in an interactive Bash session.  A session is just a series of communications between two or more entities that follows some sort of protocol (communication rules).  So, you and the computer are now communicating by the rules of Bash.  Be careful what you ask it to do, since it may well do it.  Contrast this with your children, coworkers, neighbors, pets, elected officials, and Amazon Echo.  Okay, I apologize Alexa–you sometimes do what I ask you to.

How Do I Know I Am Using Bash?

I’ll use a $ at the beginning of a sentence to indicate you should type a command in at the shell prompt.  After the command, I’ll list the expected output.  This is a common convention.  So, do this:

 $ echo $BASH_VERSION
4.1.11(2)-release

 If you see something radically different, maybe your login shell is not set to be Bash.

$ echo $SHELL
/bin/bash

 $ cat /etc/passwd | grep $USER | cut -f 7 -d':'
/bin/bash

Don’t worry if you do not understand the commands above.  I don’t actually understand them either, but that hasn’t stop me from writing this or finding gainful employment.  Just stay positive and exude confidence.  If enjoy having a basis for your confidence, read the man pages (man => manual) for the various commands.  Press ‘q’ to exit back to the shell from reading the man pages.

$ man man
$ man echo
$ man cat
$ man grep
$ man cut

If your shell is not bash, you should be able to start bash anyway (below).  Note that when I say ‘bash’ I mean the actual executing shell process on your computer.  When I say ‘Bash’, I mean the concept or language of Bash.  And when I say ‘bAsH’, it’s just a typo.  And when I put the comma outside the quotation marks, I do it intentionally.  Ask your system administrator to change your default shell to bash if desired.

$ bash

How Do I Escape from the Horrors of Bash?

You can’t.  It’s probably running on the server on every website you connect to.  Watching you.  Waiting.  Helping to serve up irrelevant ads for things you already bought.  If you’d like to try, once you’ve developed a case of carpal tunnel from all the other typing, type ‘exit’ to exit to the relative safety of Windows.

$ exit

What’s Next?

Read the Bash Guide for Beginners.  I intend to elaborate on topics in that book as I read through it myself.  Or at least I intended to in 2014 before an endless barrage of other work derailed those plans.

Here’s some historical background on the command line that you may enjoy: http://www.cryptonomicon.com/command.zip.

Keep bashing away.  It takes years to become very proficient on the command line.  It is worth it if you intend to become a highly skilled software professional.

 

Scrum: The Role of the Development Team

There are three roles in Scrum: Product Owner, Scrum Master, and Development Team.  Collectively, they form a Scrum Team.  I recently became a certified Professional Scrum Developer (PSD I).  I support Scrum because I have seen the shockingly horrible results of being forced to violate just about all of its tenants simultaneously.  I understand by the pain of experience.  Given that, I feel like it is a good time to summarize what I know about the Development Team role in Scrum in order to help other understand it more fully.

As a side note, RIP Tom Petty.  Great musician!  “I Won’t Back Down” either.

Development Team

  • Each person on the development team has only the title Developer.
    • This promotes respect (a Scrum value) by countering the ego generated by pretentious titles.
    • This promotes cross-functional teams by not assigning people specific roles like Architect, Web Developer, Database Administrator, or QA Tester.
    • You may think this is a trifle, but when pay is linked to title and people start asserting titles for which the rest of the team does not feel they are qualified, serious division and political strife occurs.
  • Is self-organizing (has autonomy).
    • Autonomy, mastery, and purpose are prime motivators above the carrot/stick level.
    • This respects the immense talent that software development professionals have.  Micromanaging such talent is like telling your doctor how to do surgery.  It’s absurdly foolish!  Yet it happens every day in poorly managed organizations.
  • Does not include the Product Owner and the Scrum Master.
    • Unless they are acting in hybrid roles (which is a very bad idea).
  • Ideally consists of 3 to 9 developers.  I believe the sweet spot is 7.
    • Less than that, and it is almost impossible to have the cross-functional skills and raw firepower necessary to build a meaningful increment of software within a sprint.
    • More than that, and subteams naturally form which undermines team unity.
    • More than that, and exponential communication overhead becomes too costly.
  • Controls the sprint backlog.  Decides how much to pull from the product backlog each sprint.
    • No one (not even the meddling, micromanaging CEO) can push work onto the sprint backlog.  Well, at least they don’t in an organization that is actually agile (rather than just claiming to be in company e-mails riddled with other lies and false promises).
    • Neither the Product Owner nor the Scrum Master can push product backlog items into the sprint backlog.  Nor can they tell the Development Team how to translate those requirements into development tasks.
  • Performs all estimates.
    • No one (not even the horrible CEO pulling fantasy deadlines out of his ass) can estimate the work to be done.  Even then, estimates are not contractual commitments to whip the peons with.  It is offensive, disrespectful, and completely ignorant for upper management to employ such tactics.
  • Is dedicated to a single project.
    • One of the Scrum principles is commitment.  If every person on your team is assigned to multiple projects, there is no commitment to any specific project.  This won’t stop the unethical, manipulative, evil CEO from beating you with the 1 project that failed even when 4 succeeded.  Being evil, he was setting you up as a scapegoat from the beginning.
  • Is collectively responsible for all development work needed to create the next software increment.
    • While one person may be the lead on a task, the team is still responsible.  This prevents team members from taking the attitude of “Well, my work is done, so I won’t be blamed if the project fails” and “I only have to not be in last place when being chased by a tiger.”
  • Collectively owns the code.
    • Man, I have seen some people get angry when someone messes up “their” code.  On my first real job, I accidentally messed up some esoteric bit in some Java code while trying to fix it and got banished to the IT room for a year.  Just for changing one line of code!  A trivial, easily-corrected problem that did not even escape into production.  Complete alpha-developer, ultra-territorial, ego-fueled, disrespectful bullshit.
  • Is cross-functional.
    • If the Development Team does not collectively have sufficient skills to produce a working increment of software, it can’t do Scrum.  This explains the hype surrounding “full stack” developers lately.
    • Skill redundancy protects against single-person bottlenecks.
      • You don’t want to be dead in the water just because your only firmware developer is on extended, indefinite leave (due to being death-marched by the CEO over Christmas).
      • Everyone needs to be able to fix the damn build.
    • In software development, the cost of context switching and context hand-offs is EXTREMELY HIGH!  The more vertical end-to-end work a single person or team can do, the better.
      • Think of it this way.  Your 400m relay team consists of Usain Bolt clones.  However, the cost of passing the baton is 1 minute.  If I, with zero athletic ability, run alone against your superstar 10x team, I will crush you into the dust every time.
      • The moral: Heroics and alleged 10x superstars do not scale in software development.  You are better off with a well-managed team of solid developers that have breadth of skill working at a sustainable pace.
  • Works at a pace that is sustainable forever.
    • Death marches are always a sign of bad management.  Always.  It’s better to 20-mile march as per Great by Choice, one of the many enlightening books I have read that helped shape my software development philosophy.
  • Has respect for each other.
    • Without respect, trust will erode which leads to transparency eroding which leads to action on false information (lack of awareness) which leads to failure.  This seems like something Yoda might say.
    • If a team member is really lacking in skills and not improving, has no commitment to following your software development practice, or has been consistently disrespectful to others, fire them immediately.  Trust me: Do not wait a single day.  It does not matter what technical skills they have.  Letting this kind of situation fester destroys the morale of the rest of the team.
  • Self-reports progress daily by burn down charts or some other publicly visible mechanism.
    • At my last job, my brother actually rigged automatic generation of sprint progress by counting checked-off “grains of sand” subtasks put into each GitHub issue.  It was great!   Before that, we had a project manager with no software experiences (our CEO justified this by the obtuse assertion that “there is no difference in doing project management for building a bridge vs. creating software”) intrusively asking us to guess percent complete for everything.  This was stupid–meaningless numbers were being reported and then altered by a person who did not understand any of the tasks.  After generating accurate progress metrics,  upper management stupidly decided they were not accurate (time proved that they totally were) and ignored them (mainly because they proved that their deadlines pulled out of thin air were complete fantasy).
    • Empirical process control is based on transparency, inspection, and adaptation.  The example above shows that transparency alone is useless if the information provided is ignored or actively suppressed.  Unethical, dishonest, incompetent CEOs try to cover up the truth so that they can lie to customers instead of working with them to create real value (merely extracting revenue via lies without creating real value is criminal–it’s called fraud).
  • Should be collocated in the same physical room if possible.
    • Yes, multinational work is more and more a reality these days.  Still, it is horribly ineffective and should be avoided whenever possible.
    • Since in-person, face-to-face communication is the ultimate (as Agile asserts), telecommuting should be used sparingly.
  • Holds the daily scrum.
    • Neither the Product Owner nor the Scrum Master is required to be here.  Nor do they run it.  Nor is the Development Team status reporting to them or anyone else in this meeting.
    • This is supposed to be a short (15 minutes maximum) meeting mainly to coordinate work and identify impediments.  The point is to inspect and adapt the daily work items to increase the odds of accomplishing the sprint goal.
    • This is not supposed to be a posturing meeting where people blow smoke or try to appear busier than others to justify their existence.
    • This is not the time for deep problem solving and lengthy technical tangents.
    • When working with remotely split teams, some virtual water cooler chat needs to happen to reduce the alienation of the remote team members.
    • This should be held at the same time and place every day to eliminate coordination costs.