tf.keras.preprocessing.text.hashing

TensorFlow 1 version

View source on GitHub

Converts a text to a sequence of words (or tokens).

View aliases

Compat aliases for migration

See Migration guide for more details.

tf.compat.v1.keras.preprocessing.text.text_to_word_sequence

tf.keras.preprocessing.text.text_to_word_sequence(
    input_text,
    filters='!"#$%&()*+,-./:;<=>?@[\\]^_`{|}~\t\n',
    lower=True, split=' '
)

This function transforms a string of text into a list of words while ignoring filters which include punctuations by default.

sample_text = 'This is a sample sentence.'
tf.keras.preprocessing.text.text_to_word_sequence(sample_text)
['this', 'is', 'a', 'sample', 'sentence']

Arguments
`input_text`	Input text (string).
`filters`	list (or concatenation) of characters to filter out, such as punctuation. Default: `'!"#$%&()*+,-./:;<=>?@[\]^_`{\|}~\t\n'`, includes basic punctuation, tabs, and newlines. </td> </tr><tr> <td>`lower`</td> <td> boolean. Whether to convert the input to lowercase. </td> </tr><tr> <td>`split`	str. Separator for word splitting.

Returns
A list of words (or tokens).

TensorFlow

tf

tf.audio

tf.autograph

tf.bitwise

tf.compat

tf.config

tf.data

tf.debugging

tf.distribute

tf.dtypes

tf.errors

tf.estimator

tf.experimental

tf.feature_column

tf.graph_util

tf.image

tf.initializers

tf.io

tf.keras

tf.linalg

tf.lite

tf.lookup

tf.losses

tf.math

tf.metrics

tf.nest

tf.nn

tf.optimizers

tf.quantization

tf.queue

tf.ragged

tf.random

tf.raw_ops

tf.saved_model

tf.sets

tf.signal

tf.sparse

tf.strings

tf.summary

tf.sysconfig

tf.test

tf.tpu

tf.train

tf.version

tf.xla

tf.keras / preprocessing / preprocessing.text.hashing_trick

View aliases

Arguments

Returns