Working with Kaldi's IO

This tutorial shows how to read and write ark/scp files in Python.

Reading and Writing Alignment Information

The following class can be used to write alignment information to files:

  • IntVectorWriter

And the following classes can be used to read alignment information from files:

  • SequentialIntVectorReader
  • RandomAccessIntVectorReader

The following code shows how to write and read alignment information.

Example of reading and writing align information
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
#!/usr/bin/env python3

import kaldi

wspecifier = 'ark,scp:/tmp/ali.ark,/tmp/ali.scp'

writer = kaldi.IntVectorWriter(wspecifier)
writer.Write(key='foo', value=[1, 2, 3])
writer.Write('bar', [10, 20])
writer.Close()

rspecifier = 'scp:/tmp/ali.scp'
reader = kaldi.SequentialIntVectorReader(rspecifier)

for key, value in reader:
    print(key, value)

reader.Close()

reader = kaldi.RandomAccessIntVectorReader(rspecifier)
value1 = reader['foo']
print(value1)

value2 = reader['bar']
print(value2)
reader.Close()

Its output is

foo [1, 2, 3]
bar [10, 20]
[1, 2, 3]
[10, 20]

The output of the following command

$ copy-int-vector scp:/tmp/ali.scp ark,t:-

is

copy-int-vector scp:/tmp/ali.scp ark,t:-
foo 1 2 3
bar 10 20
LOG (copy-int-vector[5.5.792~1-f5875b]:main():copy-int-vector.cc:83) Copied 2 vectors of int32.

5
wspecifier = 'ark,scp:/tmp/ali.ark,/tmp/ali.scp'

It creates a write specifier wspecifier indicating that the alignment information is going to be written into files /tmp/ali.ark and /tmp/ali.scp.

8
writer.Write(key='foo', value=[1, 2, 3])

It writes a list [1, 2, 3] to file with key == foo. Note that you can use keyword arguments while writing.

9
writer.Write('bar', [10, 20])

It writes a list [10, 20] to file with key == bar.

10
writer.Close()

It closes the writer.

Note

It is a best practice to close the file when it is no longer needed.

12
13
rspecifier = 'scp:/tmp/ali.scp'
reader = kaldi.SequentialIntVectorReader(rspecifier)

It creates a sequential reader.

15
16
for key, value in reader:
    print(key, value)

It uses a for loop to iterate the reader.

18
reader.Close()

It closes the reader.

20
reader = kaldi.RandomAccessIntVectorReader(rspecifier)

It creates a random access reader.

21
22
value1 = reader['foo']
print(value1)

It reads the value of foo and prints it out.

24
25
value2 = reader['bar']
print(value2)

It reads the value of bar and prints it out.

26
reader.Close()

Finally, it closes the reader.

The following code example achieves the same effect as the above one except that you do not need to close the file manually.

Example of reading and writing align information using with
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
#!/usr/bin/env python3

import kaldi

wspecifier = 'ark,scp:/tmp/ali.ark,/tmp/ali.scp'

with kaldi.IntVectorWriter(wspecifier) as writer:
    writer.Write(key='foo', value=[1, 2, 3])
    writer.Write('bar', [10, 20])

# Note that you do NOT need to close the file.

rspecifier = 'scp:/tmp/ali.scp'
with kaldi.SequentialIntVectorReader(rspecifier) as reader:
    for key, value in reader:
        print(key, value)

rspecifier = 'scp:/tmp/ali.scp'
with kaldi.RandomAccessIntVectorReader(rspecifier) as reader:
    value1 = reader['foo']
    print(value1)

    value2 = reader['bar']
    print(value2)

Reading and Writing Matrices

Using xfilename

The following code demonstrates how to read and write FloatMatrix using xfilename.

Example of reading and writing matrices with xfilename
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
#!/usr/bin/env python3

import kaldi

m = kaldi.FloatMatrix(2, 2)
m[0, 0] = 10
m[1, 1] = 20

xfilename = '/tmp/lda.mat'
kaldi.write_mat(m, xfilename, binary=True)

g = kaldi.read_mat(xfilename)
print(g)

The output of the above program is

[
 10 0
 0 20 ]

5
6
7
m = kaldi.FloatMatrix(2, 2)
m[0, 0] = 10
m[1, 1] = 20

It creates a FloatMatrix and sets its diagonal to [10, 20].

 9
10
xfilename = '/tmp/lda.mat'
kaldi.write_mat(m, xfilename, binary=True)

It writes the matrix to /tmp/lda.mat in binary format. kaldi.write_mat is used to write the matrix to the specified file. You can specify whether it is written in binary format or text format.

12
13
g = kaldi.read_mat(xfilename)
print(g)

It reads the matrix back and prints it to the console. Note that you do not need to specify whether the file to read is in binary or not. kaldi.read_mat will figure out the format automatically.

Using specifier

The following code demonstrates how to read and write FloatMatrix using specifier.

Example of reading and writing matrices with specifier
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
#!/usr/bin/env python3

import numpy as np
import kaldi

wspecifier = 'ark,scp:/tmp/feats.ark,/tmp/feats.scp'

writer = kaldi.MatrixWriter(wspecifier)

m = np.arange(6).reshape(2, 3).astype(np.float32)
writer.Write(key='foo', value=m)

g = kaldi.FloatMatrix(2, 2)
g[0, 0] = 10
g[1, 1] = 20
writer.Write('bar', g)

writer.Close()

rspecifier = 'scp:/tmp/feats.scp'
reader = kaldi.SequentialMatrixReader(rspecifier)
for key, value in reader:
    assert key in ['foo', 'bar']
    if key == 'foo':
        np.testing.assert_array_equal(value.numpy(), m)
    else:
        np.testing.assert_array_equal(value.numpy(), g.numpy())

reader.Close()

reader = kaldi.RandomAccessMatrixReader(rspecifier)
assert 'foo' in reader
assert 'bar' in reader
np.testing.assert_array_equal(reader['foo'].numpy(), m)
np.testing.assert_array_equal(reader['bar'].numpy(), g.numpy())
reader.Close()

6
7
8
wspecifier = 'ark,scp:/tmp/feats.ark,/tmp/feats.scp'

writer = kaldi.MatrixWriter(wspecifier)

This creates a matrix writer.

10
11
m = np.arange(6).reshape(2, 3).astype(np.float32)
writer.Write(key='foo', value=m)

It creates a Numpy array object of type np.float32 and writes it to file with the key foo. Note that the type of the Numpy array has to be of type np.float32. The program throws if the type is not np.float32.

13
14
15
16
g = kaldi.FloatMatrix(2, 2)
g[0, 0] = 10
g[1, 1] = 20
writer.Write('bar', g)

It creates a FloatMatrix and writes it to file with the key bar.

Hint

kaldi.MatrixWriter accepts Numpy array objects of type np.float32 as well as kaldi.FloatMatrix objects.

18
writer.Close()

It closes the writer.

20
21
rspecifier = 'scp:/tmp/feats.scp'
reader = kaldi.SequentialMatrixReader(rspecifier)

It creates a sequential matrix reader.

21
22
23
24
25
26
27
reader = kaldi.SequentialMatrixReader(rspecifier)
for key, value in reader:
    assert key in ['foo', 'bar']
    if key == 'foo':
        np.testing.assert_array_equal(value.numpy(), m)
    else:
        np.testing.assert_array_equal(value.numpy(), g.numpy())

It uses a for loop to iterate the sequential reader.

29
reader.Close()

It closes the sequential reader.

31
reader = kaldi.RandomAccessMatrixReader(rspecifier)

It creates a random access matrix reader.

32
33
assert 'foo' in reader
assert 'bar' in reader

It uses in to test whether the reader contains a given key.

34
35
np.testing.assert_array_equal(reader['foo'].numpy(), m)
np.testing.assert_array_equal(reader['bar'].numpy(), g.numpy())

It uses [] to read the value of a specified key.

36
reader.Close()

It closes the random access reader.

The following code example achieves the same effect as the above one except that you do not need to close the file manually.

Example of reading and writing FloatMatrix using with
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
#!/usr/bin/env python3

import numpy as np
import kaldi

wspecifier = 'ark,scp:/tmp/feats.ark,/tmp/feats.scp'

with kaldi.MatrixWriter(wspecifier) as writer:
    m = np.arange(6).reshape(2, 3).astype(np.float32)
    writer.Write(key='foo', value=m)

    g = kaldi.FloatMatrix(2, 2)
    g[0, 0] = 10
    g[1, 1] = 20
    writer.Write('bar', g)

rspecifier = 'scp:/tmp/feats.scp'
with kaldi.SequentialMatrixReader(rspecifier) as reader:
    for key, value in reader:
        assert key in ['foo', 'bar']
        if key == 'foo':
            np.testing.assert_array_equal(value.numpy(), m)
        else:
            np.testing.assert_array_equal(value.numpy(), g.numpy())

with kaldi.RandomAccessMatrixReader(rspecifier) as reader:
    assert 'foo' in reader
    assert 'bar' in reader
    np.testing.assert_array_equal(reader['foo'].numpy(), m)
    np.testing.assert_array_equal(reader['bar'].numpy(), g.numpy())