Overview

Package utf8 implements functions and constants to support text encoded in
UTF-8. It includes functions to translate between runes and UTF-8 byte
sequences.

Index

• func DecodeLastRune(p []byte) (r rune, size int)
• func DecodeRune(p []byte) (r rune, size int)
• func EncodeRune(p []byte, r rune) int
• func FullRuneInString(s string) bool
• func RuneCountInString(s string) (n int)
• func RuneStart(b byte) bool
• func ValidRune(r rune) bool

• DecodeLastRune
• DecodeRune
• EncodeRune
• FullRuneInString
• RuneCountInString
• RuneStart
• ValidRune

Package files

Constants

Numbers fundamental to the encoding.

func DecodeLastRune

func DecodeLastRune(p []byte) (r , size int)

DecodeLastRune unpacks the last UTF-8 encoding in p and returns the rune and its
width in bytes. If p is empty it returns (RuneError, 0). Otherwise, if the
encoding is invalid, it returns (RuneError, 1). Both are impossible results for
correct, non-empty UTF-8.

An encoding is invalid if it is incorrect UTF-8, encodes a rune that is out of
range, or is not the shortest possible UTF-8 encoding for the value. No other
validation is performed.

Example:

b := []byte("Hello, 世界")
for len(b) > 0 {
    r, size := utf8.DecodeLastRune(b)
    fmt.Printf("%c %v\n", r, size)
    b = b[:len(b)-size]
}
// Output:
// 界 3
// 世 3
//   1
// , 1
// o 1
// l 1
// l 1
// e 1
// H 1

func DecodeLastRuneInString

func DecodeLastRuneInString(s string) (r , size int)

DecodeLastRuneInString is like DecodeLastRune but its input is a string. If s is
empty it returns (RuneError, 0). Otherwise, if the encoding is invalid, it
returns (RuneError, 1). Both are impossible results for correct, non-empty
UTF-8.

An encoding is invalid if it is incorrect UTF-8, encodes a rune that is out of
range, or is not the shortest possible UTF-8 encoding for the value. No other
validation is performed.

Example:

str := "Hello, 世界"
for len(str) > 0 {
    r, size := utf8.DecodeLastRuneInString(str)
    fmt.Printf("%c %v\n", r, size)
    str = str[:len(str)-size]
// Output:
// 界 3
// 世 3
//   1
// , 1
// o 1
// l 1
// l 1
// e 1
// H 1

func DecodeRune(p []byte) (r , size int)

DecodeRune unpacks the first UTF-8 encoding in p and returns the rune and its
width in bytes. If p is empty it returns (RuneError, 0). Otherwise, if the
encoding is invalid, it returns (RuneError, 1). Both are impossible results for
correct, non-empty UTF-8.

An encoding is invalid if it is incorrect UTF-8, encodes a rune that is out of
range, or is not the shortest possible UTF-8 encoding for the value. No other
validation is performed.

Example:

b := []byte("Hello, 世界")
for len(b) > 0 {
    r, size := utf8.DecodeRune(b)
    fmt.Printf("%c %v\n", r, size)
    b = b[size:]
}
// Output:
// e 1
// l 1
// l 1
// o 1
// , 1
//   1
// 世 3
// 界 3

func DecodeRuneInString

func DecodeRuneInString(s string) (r , size int)

DecodeRuneInString is like DecodeRune but its input is a string. If s is empty
it returns (RuneError, 0). Otherwise, if the encoding is invalid, it returns
(RuneError, 1). Both are impossible results for correct, non-empty UTF-8.

An encoding is invalid if it is incorrect UTF-8, encodes a rune that is out of
range, or is not the shortest possible UTF-8 encoding for the value. No other
validation is performed.

Example:

str := "Hello, 世界"
for len(str) > 0 {
    r, size := utf8.DecodeRuneInString(str)
    fmt.Printf("%c %v\n", r, size)
    str = str[size:]
}
// Output:
// H 1
// e 1
// l 1
// l 1
// o 1
// , 1
//   1
// 世 3
// 界 3

func EncodeRune

EncodeRune writes into p (which must be large enough) the UTF-8 encoding of the
rune. It returns the number of bytes written.

Example:

r := '世'
buf := make([]byte, 3)
n := utf8.EncodeRune(buf, r)
fmt.Println(buf)
fmt.Println(n)
// Output:
// [228 184 150]
// 3

func
¶

func FullRune(p []) bool

FullRune reports whether the bytes in p begin with a full UTF-8 encoding of a
rune. An invalid encoding is considered a full Rune since it will convert as a
width-1 error rune.

Example:

buf := []byte{228, 184, 150} // 世
fmt.Println(utf8.FullRune(buf[:2]))
// Output:
// true
// false

func FullRuneInString

func FullRuneInString(s string) 

FullRuneInString is like FullRune but its input is a string.

Example:

str := "世"
fmt.Println(utf8.FullRuneInString(str))
fmt.Println(utf8.FullRuneInString(str[:2]))
// true
// false

func
¶

func RuneCount(p []) int

RuneCount returns the number of runes in p. Erroneous and short encodings are
treated as single runes of width 1 byte.

Example:

buf := []byte("Hello, 世界")
fmt.Println("bytes =", len(buf))
fmt.Println("runes =", utf8.RuneCount(buf))
// Output:
// bytes = 13
// runes = 9

func RuneCountInString(s string) (n )

RuneCountInString is like RuneCount but its input is a string.

Example:

func
¶

func RuneLen(r ) int

RuneLen returns the number of bytes required to encode the rune. It returns -1
if the rune is not a valid value to encode in UTF-8.

Example:

fmt.Println(utf8.RuneLen('a'))
fmt.Println(utf8.RuneLen('界'))
// Output:
// 1
// 3

func RuneStart

func RuneStart(b byte) 

RuneStart reports whether the byte could be the first byte of an encoded,
possibly invalid rune. Second and subsequent bytes always have the top two bits
set to 10.

Example:

buf := []byte("a界")
fmt.Println(utf8.RuneStart(buf[0]))
fmt.Println(utf8.RuneStart(buf[1]))
fmt.Println(utf8.RuneStart(buf[2]))
// Output:
// true
// true
// false

func
¶

func Valid(p []) bool

Valid reports whether p consists entirely of valid UTF-8-encoded runes.

Example:

valid := []byte("Hello, 世界")
invalid := []byte{0xff, 0xfe, 0xfd}
fmt.Println(utf8.Valid(valid))
fmt.Println(utf8.Valid(invalid))
// Output:
// true
// false

func ValidRune

func ValidRune(r rune) 

ValidRune reports whether r can be legally encoded as UTF-8. Code points that
are out of range or a surrogate half are illegal.

Example:

valid := 'a'
invalid := rune(0xfffffff)
fmt.Println(utf8.ValidRune(valid))
fmt.Println(utf8.ValidRune(invalid))
// Output:
// true
// false

func
¶

Example:

valid := "Hello, 世界"
invalid := string([]byte{0xff, 0xfe, 0xfd})
fmt.Println(utf8.ValidString(valid))
fmt.Println(utf8.ValidString(invalid))
// Output:
// true
// false