加载模型后出现警告
>>> import hanlp
>>> nlp = hanlp.load(hanlp.pretrained.dep.CTB5_BIAFFINE_DEP_ZH)
2021-11-25 15:23:08.821507: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library ‘libcudart.so.11.0’; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64
2021-11-25 15:23:08.821547: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2021-11-25 15:23:13.246129: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library ‘libcudart.so.11.0’; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64
2021-11-25 15:23:13.246908: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library ‘libcublas.so.11’; dlerror: libcublas.so.11: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64
2021-11-25 15:23:13.247432: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library ‘libcublasLt.so.11’; dlerror: libcublasLt.so.11: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64
2021-11-25 15:23:13.248193: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library ‘libcusolver.so.11’; dlerror: libcusolver.so.11: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64
2021-11-25 15:23:13.248891: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library ‘libcusparse.so.11’; dlerror: libcusparse.so.11: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64
2021-11-25 15:23:13.249405: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library ‘libcudnn.so.8’; dlerror: libcudnn.so.8: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64
2021-11-25 15:23:13.249438: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1835] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices…
2021-11-25 15:23:13.291964: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Downloading https://file.hankcs.com/hanlp/embeddings/ctb.fasttext.300.txt.zip to /home/liuzixi01/.hanlp/embeddings/ctb.fasttext.300.txt.zip
100% 31.8 MiB 6.3 MiB/s ETA: 0 s [=============================================================]
Decompressing /home/liuzixi01/.hanlp/embeddings/ctb.fasttext.300.txt.zip to /home/liuzixi01/.hanlp/embeddings
>>> MB Loading word2
然后调用模型出现错误
>>> nlp(‘杨过笑’)
2021-11-25 15:23:50.654955: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
2021-11-25 15:23:50.690515: W tensorflow/core/framework/op_kernel.cc:1680] Invalid argument: ValueError: invalid literal for int() with base 10: ‘’
Traceback (most recent call last):
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/tensorflow/python/ops/script_ops.py", line 249, in __call__
ret = func(*args)
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py", line 645, in wrapper
return func(*args, **kwargs)
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 892, in generator_py_func
values = next(generator_state.get_iterator(iterator_id))
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/hanlp/transform/conll_tf.py", line 161, in generator
yield from self.batched_inputs_to_batches(corpus, batch, shuffle)
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/hanlp/transform/conll_tf.py", line 196, in batched_inputs_to_batches
b = tf.keras.preprocessing.sequence.pad_sequences(b, padding='post',
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/keras/preprocessing/sequence.py", line 152, in pad_sequences
return sequence.pad_sequences(
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/keras_preprocessing/sequence.py", line 98, in pad_sequences
trunc = np.asarray(trunc, dtype=dtype)
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/numpy/core/_asarray.py", line 83, in asarray
return array(a, dtype, copy=False, order=order)
ValueError: invalid literal for int() with base 10: '<bos>'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/hanlp/common/component.py", line 36, in __call__
return self.predict(*args, **kwargs)
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/hanlp/common/keras_component.py", line 431, in predict
for idx, batch in enumerate(dataset):
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 761, in __next__
return self._next_internal()
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 744, in _next_internal
ret = gen_dataset_ops.iterator_get_next(
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/tensorflow/python/ops/gen_dataset_ops.py", line 2728, in iterator_get_next
_ops.raise_from_not_ok_status(e, name)
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 6941, in raise_from_not_ok_status
six.raise_from(core._status_to_exception(e.code, message), None)
File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: ValueError: invalid literal for int() with base 10: '<bos>'
Traceback (most recent call last):
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/tensorflow/python/ops/script_ops.py", line 249, in __call__
ret = func(*args)
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py", line 645, in wrapper
return func(*args, **kwargs)
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 892, in generator_py_func
values = next(generator_state.get_iterator(iterator_id))
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/hanlp/transform/conll_tf.py", line 161, in generator
yield from self.batched_inputs_to_batches(corpus, batch, shuffle)
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/hanlp/transform/conll_tf.py", line 196, in batched_inputs_to_batches
b = tf.keras.preprocessing.sequence.pad_sequences(b, padding='post',
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/keras/preprocessing/sequence.py", line 152, in pad_sequences
return sequence.pad_sequences(
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/keras_preprocessing/sequence.py", line 98, in pad_sequences
trunc = np.asarray(trunc, dtype=dtype)
File "/home/liuzixi01/.conda/envs/hanlp-cuda101/lib/python3.8/site-packages/numpy/core/_asarray.py", line 83, in asarray
return array(a, dtype, copy=False, order=order)
ValueError: invalid literal for int() with base 10: '<bos>'
[[{{node PyFunc}}]] [Op:IteratorGetNext]
求问如何解决