Going, Going, Golang II: A Rubyist’s Impression of Iris & MongoDB

The more I dug into the Go tutorial, the more I wanted to try using it out in the real world. Obviously, this is in no way motivated by my fear of the Concurrency section of the tutorial. While I’m planning to build a much larger app with it soon, I wanted to get my hands dirty building something small first. After some consideration, I settled on a Quote Generator.

Choosing a Framework

Seeing as how I was striking out into a new language for this app, I figured I should try and make everything else as familiar as possible. My initial hope was to find a framework as similar to Rails as possible: opinionated, well-featured, and easy to boot up. The less I got bogged down in the particulars, the more I could concentrate on the whole learning experience.

Having found a few resources to delve into, I decided to go with Iris. Like I’ve said, I’m still at the beginning of my career, and sticking to the more opinionated frameworks (as well as architectures that I know, cough cough MVC) feels like a safer choice for a solo undertaking. I had initially hoped for a strong generator like Rails has, so I could get a sense of common practice in project structure. Unfortunately, from the options I was given, none of them seemed to provide that functionality. Turns out, I’d have to build things the Old-Fashioned Way.

Entertainingly, I did very nearly go with Gin, because of the simplicity of my project. Because Gin could do everything I absolutely needed, it seemed like a good prospect. That being said, the project I want to work on after this will likely be more fully-featured, I should probably specialize in one for now, and branch out later if I feel limited by Iris. Ultimately, the idea of a more robust framework felt safer— it might be better to have and not need, rather than need and not have.

Choosing a Database

Since the data I planned to handle was simply a quote, a subject, and a speaker, I felt it probably unnecessary to choose a SQL approach. I could easily store everything I needed in a JSON document, removing the need to have multiple tables referencing each other. With that in mind, I decided to try using Mongodb. I’d been flirting with it at work, and I was excited to give it a try. Plus, I’d already done a little looking, and found a Mongo driver for Go. I figured connecting it all together should prove fairly straightforward.

Actually Building It

Hello World

To start easing myself into Iris, I went with their most basic Hello World functionality. And, man, was it simple to grasp.

package main
import "github.com/kataras/iris"
func main() {
  app := iris.New()
  app.Handle("GET", "/", func(ctx iris.Context) {
    ctx.HTML("<h1>Welcome</h1>")
  })         
  app.Run(iris.Addr(":8080"),
          iris.WithoutServerError(iris.ErrServerClosed))
}

In the immediate sense, it’s an extremely quick way to get up and running: make a new iris object, tell it to handle GET requests on root by serving up a given html string, then tell it to run that object on localhost:8080—with the potential to serve up an error if anything goes wrong. I don’t understand the syntax fully (namely, what appears to be an anonymous function passed into Handle that renders out the HTML), but it’s easy enough to get by.

Connecting Mongo

Next, I tried building a page that accessed Mongo. It was fairly straightforward, to implement in and of itself (with code largely adapted from Santosh Anand):

package main

import (
	"log"

	"github.com/kataras/iris"
	"gopkg.in/mgo.v2"
	"gopkg.in/mgo.v2/bson"
)

type Quote struct {
	Author  string
	Subject string
	Text    string
}

func main() {
	app := iris.Default()

	session, err := mgo.Dial("localhost")
	if nil != err {
		panic(err)
	}
	defer session.Close()
	session.SetMode(mgo.Monotonic, true)

	c := session.DB("quote-db").C("quote")
	c.Insert(&Quote{"Lajos Kossuth", "On History", "History is the revelation of providence."})

	app.Get("/", func(ctx iris.Context) {
		result := Quote{}
		err = c.Find(bson.M{"author": "Lajos Kossuth"}).One(&result)
		if err != nil {
			log.Fatal(err)
		}
		ctx.JSON(result)
	})
	app.Run(iris.Addr(":8080"))
}

Golang Gotcha: Initially, I had assumed that I could just place Go files right next to each other, in the same directory level of my repo. As it turns out, that’s definitely not the case. Each Go file needs to export its own main package, and having them in the same level causes fatal interference. I had to change my file structure from:

going-golang/
  ->simpleDB.go
  ->helloWorld.go

Into:

going-golang/
  ->simpleDB/
    ->simpleDB.go
  ->helloWorld/
    ->helloWorld.go

Not a tremendous change, but the fact that I couldn’t export two main packages in the same level wasn’t something that immediately occurred to me, as a Rubyist.

The second (much smaller) issue, was that I didn’t realize my MongoDB would be storing the keys lowercase, when they had been defined in the Quote Struct as Uppercase. There a number of times my connection exited out because I kept trying to query on Author.

Parsing the Text

I’m going to, unabashedly, pull my full list of quotes from Civilization VI’s list of quotes. Another programmer, Bret Lowrey, already managed to isolate an XML list of them from the game’s data. I’m going to take that data, parse it myself, and store it in the app’s MongoDB.

Golang Gotcha: I had a lot of trouble getting os.Open to work. I had placed quoteGenerator.go as the top level file in its own package. From there, however, I placed my input XML file into an inputs folder. Moreover, when I tried running it as a test, I was doing so from the parent folder of quoteGenerator. I’m still not 100% certain which of those conditions it was that killed my effort.

package main
import (
 "encoding/xml"
 "fmt"
 "io/ioutil"
 "log"
 "os"
)
type GameData struct {
 BaseGameText BaseGameText
}
type BaseGameText struct {
 XMLName xml.Name
 Rows    []Row `xml:"Row"`
}
type Row struct {
 Subject string `xml:"Tag,attr"`
 Text    string
}
func main() {
xmlFile, err := os.Open("quote.xml")
if err != nil {
  fmt.Println(err)
 }
defer xmlFile.Close()
byteValue, err := ioutil.ReadAll(xmlFile)
if err != nil {
  log.Fatal(err)
 }
var gameData GameData
 xml.Unmarshal(byteValue, &gameData)
for i := 0; i < len(gameData.BaseGameText.Rows); i++ {
  fmt.Println("Quote Data: " + gameData.BaseGameText.Rows[i].Text)
 }
}

Golang Gotcha: Another issue I ran into here was parsing the XML into my custom structs. Namely, I couldn’t understand the how to use the xml:"<foo>" things (I suspect they’re namespace tags?) while defining the struct. I had a lot of failed attempts before I could get it store the data properly.

Now, I needed to format the data into JSON, and ingest it into the MongoDB. This part was vastly helped by my finally figuring out how to use the debugger in VS code. Eventually, my parsing function looked like this:

type Quote struct {
 Subject string
 Text    string
 Author  string
}
func makeQuote(row Row) Quote {
x := func(c rune) bool {
  return !unicode.IsLetter(c)
 }
// Split the SUBJECT string into an array of strings, dropping any 
// non-letter character within.
strArray := strings.FieldsFunc(row.Subject, x)
// Drop the LOC_<foo> prefix found in the XML
 strArray = strArray[2:]
// Drop the _QUOTE at the end of the text
 strArray = strArray[:len(strArray)-1]
 subject := strings.Join(strArray, " ")
 subject = strings.Title(strings.ToLower(subject))
 strArray = strings.Split(row.Text, "[NEWLINE]– ")
 author := strArray[1]
 text := strArray[0]
 quote := Quote{
  Subject: subject,
  Author:  author,
  Text:    text}
 return quote
}

With that taken care of, I had my quote struct. Or at least, I thought I did. I found out, as I tried to bring it up, that Great Work quotes were structured in a fundamentally different way: with no author in the Text namespace, I needed to come up with a new way to parse them. I eventually settled on checking whether the row I was parsing was a Great Work, and, if so, would pass along a bool into my parser to ensure there were two separate logic paths. My finally makeQuote method looked like this:

func makeQuote(row Row, isGreatwork bool) Quote {
x := func(c rune) bool {
  return !unicode.IsLetter(c)
 }
strArray := strings.FieldsFunc(row.Subject, x)
 strArray = strArray[2:]
 strArray = strArray[:len(strArray)-1]
 subject := strings.Join(strArray, " ")
 subject = strings.Title(strings.ToLower(subject))
 var text string
 var author string
 if isGreatwork {
  text = row.Text
  author = subject
 } else {
  strArray = strings.Split(row.Text, "[NEWLINE]")
  text = strArray[0]
  if len(strArray) > 1 {
   author = strArray[1]
  } else {
   author = "Anonymous"
  }
 }
 text = strings.Replace(text, "”", "", -1)
 text = strings.Replace(text, "“", "", -1)
 quote := Quote{
  Subject: subject,
  Author:  author,
  Text:    text}
 return quote
}

From there, I just needed to adapt the simpleDB connection to allow me load in my quote struct. Ultimately, it came down to just modifying c.Insert(&quote{..}) into c.Insert(quote).

Pulling a Randomized Quote

Pulling a randomized quote was one of the easiest modifications to make to this.

pipe := c.Pipe([]bson.M{{"$sample": bson.M{"size": 1}}})
  resp := []bson.M{}
  err := pipe.All(&resp)
  if err != nil {
   fmt.Println("error:", err)
  }
  ctx.JSON(resp)

Again, it was relying on the mongoDB collection variable I’d set before: c:=session.DB("quote-db").C("quote”). After that, I could sit back and let mgo’s implementation of Mongo do most of the heavy lifting.

Once all that’s in place, the app is complete! Clocking in at just around 120 lines, it manages to serve up a new quote every time the page is refreshed. Once everything is assembled together, here’s what the final code looks like:

package main
import (
 "encoding/xml"
 "fmt"
 "io/ioutil"
 "log"
 "os"
 "strings"
 "unicode"
"github.com/kataras/iris"
 mgo "gopkg.in/mgo.v2"
 "gopkg.in/mgo.v2/bson"
)
type GameData struct {
 BaseGameText BaseGameText
}
type BaseGameText struct {
 XMLName xml.Name
 Rows    []Row `xml:"Row"`
}
type Row struct {
 Subject string `xml:"Tag,attr"`
 Text    string
}
type Quote struct {
 Subject string
 Text    string
 Author  string
}
func makeQuote(row Row, isGreatwork bool) Quote {
x := func(c rune) bool {
  return !unicode.IsLetter(c)
 }
strArray := strings.FieldsFunc(row.Subject, x)
 strArray = strArray[2:]
 strArray = strArray[:len(strArray)-1]
 subject := strings.Join(strArray, " ")
 subject = strings.Title(strings.ToLower(subject))
 var text string
 var author string
 if isGreatwork {
  text = row.Text
  author = subject
 } else {
  strArray = strings.Split(row.Text, "[NEWLINE]")
  text = strArray[0]
  if len(strArray) > 1 {
   author = strArray[1]
  } else {
   author = "Anonymous"
  }
 }
 text = strings.Replace(text, "”", "", -1)
 text = strings.Replace(text, "“", "", -1)
 quote := Quote{
  Subject: subject,
  Author:  author,
  Text:    text}
 return quote
}
func main() {
xmlFile, err := os.Open("quotes.xml")
if err != nil {
  fmt.Println(err)
 }
defer xmlFile.Close()
byteValue, err := ioutil.ReadAll(xmlFile)
if err != nil {
  log.Fatal(err)
 }
var gameData GameData
 var quote Quote
 xml.Unmarshal(byteValue, &gameData)
session, err := mgo.Dial("localhost")
 if nil != err {
  panic(err)
 }
 defer session.Close()
 session.SetMode(mgo.Monotonic, true)
c := session.DB("quote-db").C("quote")
for i := 0; i < len(gameData.BaseGameText.Rows); i++ {
  isGreatwork := strings.Contains(gameData.BaseGameText.Rows[i].Subject, "GREATWORK")
  quote = makeQuote(gameData.BaseGameText.Rows[i], isGreatwork)
  if err != nil {
   fmt.Println("error:", err)
  }
  c.Insert(quote)
 }
 app := iris.Default()
app.Get("/", func(ctx iris.Context) {
  pipe := c.Pipe([]bson.M{{"$sample": bson.M{"size": 1}}})
  resp := []bson.M{}
  err := pipe.All(&resp)
  if err != nil {
   fmt.Println("error:", err)
  }
  ctx.JSON(resp)
 })
app.Run(iris.Addr(":8080"))
}

Conclusions

Phew! That ended up being a bit more work than I initially anticipated. It was an exciting experiment, to be sure, and it’s given me a very interesting taste of Go development, overall.

It’s a fundamentally different beast from developing in Rails, to be sure. A number of the Gotchas I encountered had to do with project configuration— a realm completely swept away by that good old Rails Magic. I still need to figure out whether it’s possible to fold multiple packages into one project. So far, I’ve only been able to work with a main package, into which I have to cram all my functionality.

Another thing I found different from Ruby was how differently it treated objects. Instead of being able to call any method I wanted on any of my structs, I found that many functions took my structs as arguments, instead of being called on them as receivers. Now, that may be a result of my own amateur nature, but I found it noteworthy nonetheless.

Next Steps

So, what’s next for Going, Going, Golang? I think top priority right now is figuring out how to deploy Quote Generator. I’m thinking of just using Heroku, to keep things simple, but my first attempts to get it on there met with failure. Once I can get it up there, however, I’m planning on building and deploying a fuller app with Iris. I’ll keep writing more posts the closer I get, so be sure to check back in!