HUGO
Menu
GitHub 90206 stars Mastodon

transform.Unmarshal

Returns a map or a slice parsed from serialized data in CSV, JSON, TOML, YAML, or XML format.
Syntax
transform.Unmarshal [OPTIONS] INPUT
Returns
any
Alias
unmarshal

Usage

The transform.Unmarshal function returns a map or a slice parsed from serialized data in CSV, JSON, TOML, YAML, or XML format. The INPUT can be a string or a resource.

The return type depends on the format and the data. Unmarshaling an object returns a map, and unmarshaling an array returns a slice. Unmarshaling CSV data returns a slice of rows, as described in working with CSV.

When the INPUT is a resource, Hugo caches the result, so calling the function multiple times with the same resource incurs no additional overhead.

Options

The transform.Unmarshal function accepts an options map.

delimiter
(string) Applicable to CSV files. The delimiter used. Default is ,.
comment
(string) Applicable to CSV files. The comment character used in the CSV. If set, lines beginning with the comment character without preceding whitespace are ignored.
format
New in v0.149.0
(string) The serialization format of the input, one of csv, json, org, toml, xml, or yaml. If empty or unspecified, Hugo infers the format from the input. For resources, this option is only needed if the file lacks an extension or to override the inferred format. For strings, it’s only required when the format is ambiguous.
lazyQuotes
(bool) Applicable to CSV files. Whether to allow a quote in an unquoted field, or to allow a non-doubled quote in a quoted field. Default is false.
targetType
New in v0.146.7
(string) Applicable to CSV files. The target data type, either slice or map. Default is slice.

Examples

{{ $string := `
title: Les Misérables
author: Victor Hugo
`}}

{{ $book := transform.Unmarshal $string }}
{{ $book.title }} → Les Misérables
{{ $book.author }} → Victor Hugo

The examples below unmarshal global, page, and remote resources, then show how to work with CSV and XML data.

Global resource

A global resource is a file within the assets directory, or within any directory mounted to the assets directory.

assets/
└── data/
    └── books.json
{{ $data := dict }}
{{ $path := "data/books.json" }}
{{ with resources.Get $path }}
  {{ with . | transform.Unmarshal }}
    {{ $data = . }}
  {{ end }}
{{ else }}
  {{ errorf "Unable to get global resource %q" $path }}
{{ end }}

{{ range where $data "author" "Victor Hugo" }}
  {{ .title }} → Les Misérables
{{ end }}

Page resource

A page resource is a file within a page bundle.

content/
├── post/
│   └── book-reviews/
│       ├── books.json
│       └── index.md
└── _index.md
{{ $data := dict }}
{{ $path := "books.json" }}
{{ with .Resources.Get $path }}
  {{ with . | transform.Unmarshal }}
    {{ $data = . }}
  {{ end }}
{{ else }}
  {{ errorf "Unable to get page resource %q" $path }}
{{ end }}

{{ range where $data "author" "Victor Hugo" }}
  {{ .title }} → Les Misérables
{{ end }}

Remote resource

A remote resource is a file on a remote server, accessible via HTTP or HTTPS.

{{ $data := dict }}
{{ $url := "https://example.org/books.json" }}
{{ with try (resources.GetRemote $url) }}
  {{ with .Err }}
    {{ errorf "%s" . }}
  {{ else with .Value }}
    {{ $data = . | transform.Unmarshal }}
  {{ else }}
    {{ errorf "Unable to get remote resource %q" $url }}
  {{ end }}
{{ end }}

{{ range where $data "author" "Victor Hugo" }}
  {{ .title }} → Les Misérables
{{ end }}

When retrieving remote data, a misconfigured server may send a response header with an incorrect Content-Type. For example, the server may set the Content-Type header to application/octet-stream instead of application/json.

In these cases, pass the resource Content through the transform.Unmarshal function instead of passing the resource itself. For example, in the remote resource example above, use this instead:

{{ $data = .Content | transform.Unmarshal }}

Working with CSV

Unmarshaling CSV data returns a slice of rows. Every value is a string, even if it looks like a number. The targetType option determines the type of each row:

  • With the default slice target, each row is a slice of values. The first row contains the header.
  • With the map target, each row is a map keyed by the header. The result does not include the header row. The CSV data must have a header row and at least one data row, and the header must not contain duplicate field names.

The examples below use this CSV file:

"name","type","breed","age"
"Spot","dog","Collie",3
"Rover","dog","Boxer",5
"Felix","cat","Calico",7

To render an HTML table from a CSV file:

{{ $data := slice }}
{{ $file := "pets.csv" }}
{{ with or (.Resources.Get $file) (resources.Get $file) }}
  {{ $data = transform.Unmarshal . }}
{{ end }}

{{ with $data }}
  <table>
    <thead>
      <tr>
        {{ range index . 0 }}
          <th>{{ . }}</th>
        {{ end }}
      </tr>
    </thead>
    <tbody>
      {{ range . | after 1 }}
        <tr>
          {{ range . }}
            <td>{{ . }}</td>
          {{ end }}
        </tr>
      {{ end }}
    </tbody>
  </table>
{{ end }}

To filter or sort the rows by field name, set the targetType option to map:

{{ $data := slice }}
{{ $file := "pets.csv" }}
{{ with or (.Resources.Get $file) (resources.Get $file) }}
  {{ $opts := dict "targetType" "map" }}
  {{ $data = transform.Unmarshal $opts . }}
{{ end }}

{{ with sort (where $data "type" "dog") "name" "asc" }}
  <table>
    <thead>
      <tr>
        <th>name</th>
        <th>type</th>
        <th>breed</th>
        <th>age</th>
      </tr>
    </thead>
    <tbody>
      {{ range . }}
        <tr>
          <td>{{ .name }}</td>
          <td>{{ .type }}</td>
          <td>{{ .breed }}</td>
          <td>{{ .age }}</td>
        </tr>
      {{ end }}
    </tbody>
  </table>
{{ end }}

To compare a value as a number, convert it with the cast.ToInt or cast.ToFloat function. To list the pets older than five:

{{ range $data }}
  {{ if gt (int .age) 5 }}
    {{ .name }}
  {{ end }}
{{ end }}

Working with XML

When unmarshaling an XML file, do not include the root node when accessing data. For example, after unmarshaling the RSS feed below, access the feed title with $data.channel.title.

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Books on Example Site</title>
    <link>https://example.org/books/</link>
    <description>Recent content in Books on Example Site</description>
    <language>en-US</language>
    <atom:link href="https://example.org/books/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>The Hunchback of Notre Dame</title>
      <description>Written by Victor Hugo</description>
      <link>https://example.org/books/the-hunchback-of-notre-dame/</link>
      <pubDate>Mon, 09 Oct 2023 09:27:12 -0700</pubDate>
      <guid>https://example.org/books/the-hunchback-of-notre-dame/</guid>
    </item>
    <item>
      <title>Les Misérables</title>
      <description>Written by Victor Hugo</description>
      <link>https://example.org/books/les-miserables/</link>
      <pubDate>Mon, 09 Oct 2023 09:27:11 -0700</pubDate>
      <guid>https://example.org/books/les-miserables/</guid>
    </item>
  </channel>
</rss>

Get the remote data:

{{ $data := dict }}
{{ $url := "https://example.org/books/index.xml" }}
{{ with try (resources.GetRemote $url) }}
  {{ with .Err }}
    {{ errorf "%s" . }}
  {{ else with .Value }}
    {{ $data = . | transform.Unmarshal }}
  {{ else }}
    {{ errorf "Unable to get remote resource %q" $url }}
  {{ end }}
{{ end }}

Inspect the data structure:

<pre>{{ debug.Dump $data }}</pre>

List the book titles:

{{ with $data.channel.item }}
  <ul>
    {{ range . }}
      <li>{{ .title }}</li>
    {{ end }}
  </ul>
{{ end }}

Hugo renders this to:

<ul>
  <li>The Hunchback of Notre Dame</li>
  <li>Les Misérables</li>
</ul>

This RSS feed adds a lang attribute to each title node, and a namespaced node for the ISBN number:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0"
  xmlns:atom="http://www.w3.org/2005/Atom"
  xmlns:isbn="http://schemas.isbn.org/ns/1999/basic.dtd"
>
  <channel>
    <title>Books on Example Site</title>
    <link>https://example.org/books/</link>
    <description>Recent content in Books on Example Site</description>
    <language>en-US</language>
    <atom:link href="https://example.org/books/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title lang="en">The Hunchback of Notre Dame</title>
      <description>Written by Victor Hugo</description>
      <isbn:number>9780140443530</isbn:number>
      <link>https://example.org/books/the-hunchback-of-notre-dame/</link>
      <pubDate>Mon, 09 Oct 2023 09:27:12 -0700</pubDate>
      <guid>https://example.org/books/the-hunchback-of-notre-dame/</guid>
    </item>
    <item>
      <title lang="fr">Les Misérables</title>
      <description>Written by Victor Hugo</description>
      <isbn:number>9780451419439</isbn:number>
      <link>https://example.org/books/les-miserables/</link>
      <pubDate>Mon, 09 Oct 2023 09:27:11 -0700</pubDate>
      <guid>https://example.org/books/les-miserables/</guid>
    </item>
  </channel>
</rss>

After retrieving the remote data, inspect the data structure:

<pre>{{ debug.Dump $data }}</pre>

Each item node looks like this:

{
  "description": "Written by Victor Hugo",
  "guid": "https://example.org/books/the-hunchback-of-notre-dame/",
  "link": "https://example.org/books/the-hunchback-of-notre-dame/",
  "number": "9780140443530",
  "pubDate": "Mon, 09 Oct 2023 09:27:12 -0700",
  "title": {
    "#text": "The Hunchback of Notre Dame",
    "-lang": "en"
  }
}

The title keys do not begin with an underscore or a letter, so they are not valid identifiers. Use the collections.Index function to access the values:

{{ with $data.channel.item }}
  <ul>
    {{ range . }}
      {{ $title := index .title "#text" }}
      {{ $lang := index .title "-lang" }}
      {{ $ISBN := .number }}
      <li>{{ $title }} ({{ $lang }}) {{ $ISBN }}</li>
    {{ end }}
  </ul>
{{ end }}

Hugo renders this to:

<ul>
  <li>The Hunchback of Notre Dame (en) 9780140443530</li>
  <li>Les Misérables (fr) 9780451419439</li>
</ul>