Golang Unmarshal Variable Typed JSON Fields

This article talks about decoding/unmarshalling JSON data that may have different types for the same field. For instance, if your application is accepting data from multiple sources and a field may arrive as either a int or as a string. How do you unmarshal a field that could be either an int or a string?   How do you unmarshal an enum that arrives a string? The short answer is that we unmarshal to an intermediary type using interface{}.  

Let's start with the final type that would be used by our application:

type TradeSide int32

const (
	Ask TradeSide = iota
	Bid
)

// Trade is our final type that is used by the application.
// Instead of directly converting from JSON we will use a
// custom unmarshaller since our data may arrive in an slightly
// different formats from different sources.
type Trade struct {
	Exchange string
	Base     string
	Quote    string
	TradeID  string
	Unix     uint64
	Side     TradeSide
	Price    float64
	Amount   float64
}

This type specifies that TradeId should be a string, it uses an enum for Side and uses float64 for Price and Amount.

You may also notice that the there are no struct tags for JSON. This is because we will define an anonymous struct that does use struct tags to perform the JSON deserialization.  This looks like:

raw := struct {
	Exchange string      `json:"exchange"`
	Base     string      `json:"base"`
	Quote    string      `json:"quote"`
	TradeID  interface{} `json:"tradeId"`
	Unix     uint64      `json:"unix"`
	Side     string      `json:"side"`
	Price    interface{} `json:"price"`
	Amount   interface{} `json:"amount"`
}{}

For fields that could be various types we can use interface{} to put whatever data happens to unmarshal into it. This gets used in json.Unmarshal :

err := json.Unmarshal(bytes, &raw)
if err != nil {
	// handle err
}

Now that we have data stored in our raw, we need to decode the values into our final struct. We do this by switching on the type of the value store in raw.TradeId, raw.Side, raw.Price, or raw.Amount.

For instance we can check TradeId:

switch v := raw.TradeID.(type) {
case float64:
	trade.TradeID = strconv.Itoa(int(v))
case string:
	trade.TradeID = v
}

In Go, numeric types are always float64. We want our final TradeID value to be a string. So our code checks if it is a numeric type and converts it to a string. If it's already a string then we don't need to do anything else.

If we want to convert a string value Side into our enum values we can perform a similar action:

switch raw.Side {
case "ask", "sell":
	trade.Side = Ask
case "bid", "buy":
	trade.Side = Bid
}

Here the values could be either ask or sell and we use the enum value Ask.

That's pretty much all there is to it. Below is a fuly implemented version that processes all of our variable fields.

// UnmarshalTrade converts json bytes into a Trade instance and
// is flexible about how it handles json fields which may have
// different values
func UnmarshalTrade(bytes []byte) (*Trade, error) {
	// Construct an anonymous struct that has looser typing
	// than our output field. We use this as a temporarily
	// placeholder to parse the contents and construct
	// a properly constructed final result
	raw := struct {
		Exchange string      `json:"exchange"`
		Base     string      `json:"base"`
		Quote    string      `json:"quote"`
		TradeID  interface{} `json:"tradeId"`
		Unix     uint64      `json:"unix"`
		Side     string      `json:"side"`
		Price    interface{} `json:"price"`
		Amount   interface{} `json:"amount"`
	}{}
	err := json.Unmarshal(bytes, &raw)
	if err != nil {
		return nil, err
	}

	// Construct our Trade instance with as much information as
	// possible from the raw data
	trade := &Trade{
		Exchange: raw.Exchange,
		Base:     raw.Base,
		Quote:    raw.Quote,
		Unix:     raw.Unix,
	}

	// Populate TradeId by converting the value into a string
	// depending on the type of the value received
	switch v := raw.TradeID.(type) {
	case float64:
		trade.TradeID = strconv.Itoa(int(v))
	case string:
		trade.TradeID = v
	}

	// Populate the Side property, which will either base
	switch raw.Side {
	case "ask", "sell":
		trade.Side = Ask
	case "bid", "buy":
		trade.Side = Bid
	}

	// Populate Price by converting the value into a float
	// depending on the type received in JSON
	switch v := raw.Price.(type) {
	case float64:
		trade.Price = v
	case string:
		{
			p, err := strconv.ParseFloat(string(v), 64)
			if err != nil {
				return nil, err
			}
			trade.Price = p
		}
	}
	
	// Populate Amount by converting the value into a float
	// depending on the type received in JSON
	switch v := raw.Amount.(type) {
	case float64:
		trade.Amount = v
	case string:
		{
			p, err := strconv.ParseFloat(string(v), 64)
			if err != nil {
				return nil, err
			}
			trade.Amount = p
		}
	}

	return trade, nil
}

You can see a full version below or give it a shot in the playground: https://play.golang.org/p/dD3BnizCyxH

Decoding Arbitrary Data

Lastly, if we wanted to get super crazy instead of unmarshalling to an intermediary type, we could instead unmarshal to an interface{}:

var raw interface{}
err := json.Unmarshal(bytes, &raw)
if err != nil {
	// handle err
}

However, this requires us to access things via a map of type map[string]interface{}.  If you're curious about this check out the "Decoding Arbitrary Data" section of JSON and Go.