Cfgmgmgtcamp 2019

1. Copyright © 2019
HashiCorp
I Don't Know
What I'm Doing:
A Newbie Guide for
Golang for DevOps
Feb 5th 2019

2. @petersouter
“Technical Account Manager at HashiCorp
Peter Souter
Based in...
London, UK
This is my 4th CfgMgmentCamp
Been coming since 2014 (missed last year due
to baby!)
Worn a lot of hats in my time...
Developer, Consultant, Pre-Sales, TAM
Interested in...
Making people’s operational life easier and
more secure
DEVOPS ALL THE THINGS
Introductions - Who is this guy?
@petersouter

3. “
A Quick Primer on Golang
“The Go programming language was conceived in late 2007 as an
answer to some of the problems we were seeing developing software
infrastructure at Google.
[…]
Go was designed and developed to make working in [modern
infrastructure] more productive.
Besides its better-known aspects such as built-in concurrency and
garbage collection, Go's design considerations include rigorous
dependency management, the adaptability of software architecture as
systems grow, and robustness across the boundaries between
components.”
https://talks.golang.org/2012/splash.article#TOC_1
@petersouter

4. “
How did Golang get so popular in a DevOps world?
▪ Compiled binary
▪ Platform independent
▪ Static type checking
▪ Extremely fast
▪ Excellent CLI libraries
▪ Everyone else was doing it!
Simon Waldherr [CC BY-SA 4.0], from Wikimedia Commons
@petersouter

5. “ Docker
Kubernetes
Etcd
Prometheus
Grafana
Juju
Tsuru
A few examples
@petersouter

6. “
HashiCorp <3 Go
Terraform
Vault
Consul
Nomad
Packer
Vagrant (Coming soon…?)
@petersouter

7. “
Another quick quote...
"Why Go?" I've talked a lot about this and for HashiCorp in particular
"Everyone else is doing it" wasn't a thing: Docker wasn't out yet,
CoreOS didn't exist, etc.
We chose Go through process of elimination. Also, I've said before that
Go is painfully pragmatic. Is it the safest? Nope. Is it the fastest? Nope.
Is it the biggest ecosystem? Nope. etc. But its damn "good enough"
at just about everything. And that's big!
- Mitchell Hashimoto
@petersouter

8. “
HashiCorp has a _lot_ of Go libraries
@petersouter

9. “
So finally after years of playing around...
“Ok, I should probably try this
stuff out then...”
@petersouter

10. “
Who else here is a “glue developer”?
More of an Ops background than Dev
Rubyist/Pythonista/Perler
Experience is more on scripting
Can PR a “good-ish” fix if needed!
@petersouter

11. “
Where am I at now?
“I know enough to get into
trouble”
@petersouter

12. “
Also known as...
@petersouter

13. “
Good things to know...
We don’t have time to go through
all the minutiae of Golang but let's
talk about the things that stuck out
to me the most...
@petersouter

14. “
Comparing two popular languages...
A quick primer of basics for
Golang vs Ruby
With help from
https://medium.com/devnetwork/golang-for-ruby-developers-c0a0ce19e3677
and
https://gobyexample.com/
@petersouter

15. “ // Golang
var a int = 1
// OR
// this dynamically declares
// type for variable a as int.
var a = 1
// this dynamically defines
variable a // and declares its
type as int.
a := 1
Golang vs Ruby: Variables
#Ruby
# since Ruby is
#dynamically typed we
#cant force a type
a = 1
# unless we specify the
creation method
a = Integer(1)
a.class
=> Integer
@petersouter

16. “
Golang vs Ruby: Types re-assignment
#Ruby
a = 1
a = "Hello"
a := 1
a := 2
// fine!
a = "Hello"
// not fine!
: cannot use "hello" (type
string) as type int is
assignment
@petersouter

17. “
Golang vs Ruby: Hash/Map
#Ruby:
hash = {conf_name:
'cfgmgmtcamp', location:
'Ghent'}
# access location
location =
hash[:location]
//Go
hash := map[string]string{"conf_name":
"cfgmgmtcamp", "location": "Ghent"}
// access location
location := hash["location"]
@petersouter

18. “
Golang vs Ruby: Arrays
#Ruby
array = [1,2,3]
//Go
array := [3]int{1,2,3}
// or if we’re lazy, let the compiler
// count for us!
array := [...]int{1,2,3}
@petersouter

19. “
Slices: Unindexed arrays
package main
import "fmt"
func main() {
s := make([]string, 3)
fmt.Println("empty on creation:", s)
s[0] = "a"
s[1] = "b"
s[2] = "c"
// s[3] = "d" - Wont work: panic: runtime error: index
out of range
fmt.Println("added elements manually to limit of 3:", s)
// check length
fmt.Println("Length before append is:", len(s))
s = append(s, "d")
fmt.Println("Length after append is:", len(s))
fmt.Println("Elements after append:", s)
}
empty on creation: [ ]
added elements manually to limit of 3: [a b c]
Length before append is: 3
Length after append is: 4
Elements after append: [a b c d]
Adapted from
https://gobyexample.com/slices
[CC BY 3.0]
@petersouter

20. “
Slices: Memory Management
package main
import (
"io/ioutil"
"regexp"
"fmt"
)
var digitRegexp = regexp.MustCompile("[0-9]+")
// ruh roh, the returned slice points into an array containing the entire file!
func FindDigits(filename string) []byte {
b, _ := ioutil.ReadFile(filename)
return digitRegexp.Find(b)
}
// instead: copy the interesting data to a new slice before returning it.
// old array gets garbage collected
func CopyDigits(filename string) []byte {
b, _ := ioutil.ReadFile(filename)
b = digitRegexp.Find(b)
c := make([]byte, len(b))
copy(c, b)
return c
}
@petersouter

21. “
Golang vs Ruby: Unused variables
# Ruby
a = [1,2,3]
b = “123”
puts b
# No problem with unused
# variables in Ruby!
sum := 0
a := [3]int{1,2,3}
for index, num := range a {
sum += num
}
// not fine!
: index declared and not used
for _, n := range a {
sum += n
}
// a-ok!
pathFile, _ := os.Stat(path)
// Bad! This code will crash if path does not exist.
if pathFile.IsDir() {
fmt.Printf("%s is a directory!n", path)
}
@petersouter

22. “
Errors: err != nil
result, err := SomethingNeat()
if err != nil {
return nil, err
}
another, err := MoreNeatStuff(foo)
if err != nil {
return nil, err
}
another, err := AndAgain(baz)
if err != nil {
return nil, err
}
@petersouter

23. “
Errors: err != nil
@petersouter

24. “
Errors: err != nil
@petersouter

25. “
Errors: Don’t worry about it...
“After you've been programming in Go a while you'll appreciate that
having to check the error for every function makes you think about
what it actually means if that function goes wrong and how you should
be dealing with it.”
-Nick Craig Wood
@petersouter

26. “
Errors: Slightly nicer
if err := SomethingNeat(); err != nil {
return err
}
@petersouter

27. “
Errors: Giving More Context
type SyntaxError struct {
msg string // description of error
Offset int64 // error after reading Offset bytes
}
func (e *SyntaxError) Error() string {
return e.msg
}
if err := dec.Decode(&val); err != nil {
if serr, ok := err.(*json.SyntaxError); ok {
line, col := findLine(f, serr.Offset)
return fmt.Errorf("%s:%d:%d: %v", f.Name(), line, col, err)
}
return err
}
@petersouter

28. “
Errors: DRY-ing multiple errors
import "fmt"
func validError(errs ...error) error {
for i, _ := range errs {
if errs[i] != nil {
return errs[i]
}
}
return nil
}
func doSomething(x int) (int, error) {
if x == 2 {
return 0, fmt.Errorf("Oh noes: %d", x)
}
return x, nil
}
func handled() error {
x, err1 := doSomething(2)
y, err2 := doSomething(3)
if e := validError(err1, err2); e != nil {
return e
}
fmt.Println(x, y)
return nil
}
Adapted from
https://stackoverflow.com/questi
ons/18771569/avoid-checking-if-
error-is-nil-repetition
@petersouter

29. “
Errors: The Future in Go 2.0 (check/handle)
// old way
func printSum(a, b string) error {
x, err := strconv.Atoi(a)
if err != nil {
return err
}
y, err := strconv.Atoi(b)
if err != nil {
return err
}
fmt.Println("result:", x + y)
return nil
}
// new way
func printSum(a, b string) error {
handle err { return err }
x := check strconv.Atoi(a)
y := check strconv.Atoi(b)
fmt.Println("result:", x + y)
return nil
}
@petersouter

30. “
Best IDE for Go beginners IMHO?
Visual Code Studio
Or
GoLand
@petersouter

31. “
VSC - Automatically add imports
@petersouter

32. “
VSC - Automatic test macros
@petersouter

33. “
Lastly...packaging
• go mod actually simplifies a lot!
• No longer have to worry about $GOPATH
• go mod init converts from previous
approaches (Glide, Dep, Govendor etc)
• Otherwise, look into dep which was the
previously blessed approach
@petersouter

34. “
We only have 25 mins...
• Testing
• Makefiles
• JSON Marshaling
• Deferring
• Pointers
• Mutexes
• Channels
@petersouter

35. “I wanted to add some missing fields to the GCP
auth backend in the Vault Terraform provider:
• "bound_zones"
• "bound_regions"
• "bound_instance_groups"
• "Bound_labels"
Some help for starting on TF development
https://www.terraform.io/docs/extend/writing-custom-providers.html)
Real life example: Vault Terraform
Provider
@petersouter

36. “
Ok, lets look at the existing code:
https://github.com/terraform-providers/terraform-provider-vault/pull/227/files
"bound_service_accounts": &schema.Schema{
Type: schema.TypeSet,
Elem: &schema.Schema{
Type: schema.TypeString,
},
Optional: true,
Computed: true,
},

37. “
Ok, lets look at the existing code:
https://github.com/terraform-providers/terraform-provider-vault/pull/227/files
"bound_zones": &schema.Schema{
Type: schema.TypeSet,
Elem: &schema.Schema{
Type: schema.TypeString,
},
Optional: true,
Computed: true,
},

38. “
Now how do we actually do a lookup?
https://github.com/terraform-providers/terraform-provider-vault/pull/227/files
if v, ok := d.GetOk("bound_service_accounts"); ok {
data["bound_service_accounts"] = v.(*schema.Set).List()
}

39. “
Now how do we actually do a lookup?
https://github.com/terraform-providers/terraform-provider-vault/pull/227/files
if v, ok := d.GetOk("bound_zones"); ok {
data["bound_zones"] = v.(*schema.Set).List()
}

40. “
Yay! We’re a go Developer now!
https://github.com/terraform-provi
ders/terraform-provider-vault/pull/
227
Merged! 🎉

41. “
But we were on easy mode...
But really we didn’t have to flex our go muscles
too much…
A lot of the existing work guided us…
Lets start from scratch and flex a little!

42. “
Project - A git helper CLI app
I want a simple git cli app to do
some basic cleanup of repos

43. “
List branches that have already
been merged into master, then give
the option to delete the branch
Project - gitsweeper

44. dir, err := os.Getwd()
if err != nil {
log.Fatalf("Error opening current directory: %s", err)
return
}
repo, err := git.PlainOpen(dir)
if err != nil {
log.Fatalf("Error reading %s as Git repo: %s", dir, err)
return
}
branchRefs, err := repo.Branches()
if err != nil {
log.Fatalf("list branches failed: %s", err)
}

45. fmt.Printf("There are %d branchesn", len(branchHeads))
for branchName, branchHead := range branchHeads {
fmt.Printf("Branch %s head is: %sn", branchName, branchHead)
}
nonMasterBranchRefs := branchHeads
delete(nonMasterBranchRefs, "master")
masterCommits, err := repo.Log(&git.LogOptions{From: branchHeads["master"]})
err = masterCommits.ForEach(func(commit *object.Commit) error {
for branchName, branchHead := range nonMasterBranchRefs {
if branchHead.String() == commit.Hash.String() {
fmt.Printf("Branch %s head (%s) was found in master, so has been merged!n", branchName, branchHead)
}
}
return nil
})

46. “
Ta-dah! All working!
https://github.com/petems/gitsweeper
Now I’m a golang developer 👨💻

47. “
Conclusion
▪ Go is a great language for a
devops world, and well worth
learning
▪ It’s not as hard to learn as you
think - If I can do it, anyone can!
▪ Get your hands dirty: try making
a CLI app or submit patches to
your favourite tools
▪ There’s a ton of great resources
out there!

48. “
Great Links
▪ https://gobyexample.com/
▪ https://tour.golang.org/welcome/1
▪ https://www.openmymind.net/The-Little-Go-Book/
▪ http://www.golang-book.com/books/intro
▪ https://www.terraform.io/docs/extend/writing-custom-providers.ht
ml
▪ https://www.hashicorp.com/resources/creating-terraform-provider-f
or-anything

49. Thank you.
hello@hashicorp.comwww.hashicorp.com