ImportError:无法从"sklearn.externals"导入名称"joblib"

问题描述

我正在尝试使用joblib从s3加载我保存的模型

I am trying to load my saved model from s3 using joblib

import pandas as pd
import numpy as np
import json
import subprocess
import sqlalchemy
from sklearn.externals import joblib

ENV = 'dev'
model_d2v = load_d2v('model_d2v_version_002', ENV)

def load_d2v(fname, env):
    model_name = fname
    if env == 'dev':
        try:
            model=joblib.load(model_name)
        except:
            s3_base_path='s3://sd-flikku/datalake/doc2vec_model'
            path = s3_base_path+'/'+model_name
            command = "aws s3 cp {} {}".format(path,model_name).split()
            print('loading...'+model_name)
            subprocess.call(command)
            model=joblib.load(model_name)
    else:
        s3_base_path='s3://sd-flikku/datalake/doc2vec_model'
        path = s3_base_path+'/'+model_name
        command = "aws s3 cp {} {}".format(path,model_name).split()
        print('loading...'+model_name)
        subprocess.call(command)
        model=joblib.load(model_name)
    return model

但是我遇到了这个错误:

But i was facing this error:

    from sklearn.externals import joblib
ImportError: cannot import name 'joblib' from 'sklearn.externals' (C:\Users\prane\AppData\Local\Programs\Python\Python37\lib\site-packages\sklearn\externals\__init__.py)

然后我尝试通过直接安装joblib

Then i tried installing joblib directly by doing

import joblib

但是它给了我这个错误

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 8, in load_d2v_from_s3
  File "/home/ec2-user/.local/lib/python3.7/site-packages/joblib/numpy_pickle.py", line 585, in load
    obj = _unpickle(fobj, filename, mmap_mode)
  File "/home/ec2-user/.local/lib/python3.7/site-packages/joblib/numpy_pickle.py", line 504, in _unpickle
    obj = unpickler.load()
  File "/usr/lib64/python3.7/pickle.py", line 1088, in load
    dispatch[key[0]](self)
  File "/usr/lib64/python3.7/pickle.py", line 1376, in load_global
    klass = self.find_class(module, name)
  File "/usr/lib64/python3.7/pickle.py", line 1426, in find_class
    __import__(module, level=0)
ModuleNotFoundError: No module named 'sklearn.externals.joblib'

您能告诉我如何解决吗?预先感谢

Can you tell me how to solve this? Thanks in advance

推荐答案

您现有的泡菜保存文件(model_d2v_version_002)似乎在非标准位置(sklearn.externals.joblib中的joblib)编码了一个参考模块.而不是顶层.

It looks like your existing pickle save file (model_d2v_version_002) encodes a reference module in a non-standard location – a joblib that's in sklearn.externals.joblib rather than at top-level.

当前的scikit-learn文档仅讨论顶级joblib –例如，在 3.4.1持久性示例 –但是我确实在其他人的旧问题中看到了引用scikit-learn 0.21版中的DeprecationWarning 关于一个较旧的scikit.external.joblib变体将消失的消息:

The current scikit-learn documentation only talks about a top-level joblib – eg in 3.4.1 Persistence example – but I do see a reference in someone else's old issue to a DeprecationWarning in scikit-learn version 0.21 about an older scikit.external.joblib variant going away:

弃用"表示将某些内容标记为不建议依赖，因为它可能会在将来的版本中停止使用(通常，但并非总是如此，建议使用更新的相同方法来进行处理).

'Deprecation' means marking something as inadvisable to rely-upon, as it is likely to be discontinued in a future release (often, but not always, with a recommended newer way to do the same thing).

我怀疑您的model_d2v_version_002文件是从scikit-learn的旧版本中保存的，并且您现在使用的是scikit-learn(又名sklearn)0.23+版本，该版本已完全删除了sklearn.external.joblib变体.因此，您的文件无法直接或轻松地加载到当前环境中.

I suspect your model_d2v_version_002 file was saved from an older version of scikit-learn, and you're now using scikit-learn (aka sklearn) version 0.23+ which has totally removed the sklearn.external.joblib variation. Thus your file can't be directly or easily loaded to your current environment.

但是，对于DeprecationWarning，您可能可以暂时使用较旧的scikit-learn版本以旧方式一次加载文件，然后以现在首选的方式重新保存.给定警告信息，这可能需要scikit-learn版本0.21.x或0.22.x，但是如果您确切地知道保存model_d2v_version_002文件所使用的版本，则可以尝试使用该版本.步骤大致为:

But, per the DeprecationWarning, you can probably temporarily use an older scikit-learn version to load the file the old way once, then re-save it with the now-preferred way. Given the warning info, this would probably require scikit-learn version 0.21.x or 0.22.x, but if you know exactly which version your model_d2v_version_002 file was saved from, I'd try to use that. The steps would roughly be:

• 使用较早的sklearn

do导入类似的内容:

do imports something like:

import sklearn.external.joblib as extjoblib
import joblib

• extjoblib.load()按照您的计划，您的旧文件，但是立即使用顶级joblib重新joblib.dump(). (您可能想使用一个不同的名称，以保留较旧的文件，以防万一.)

  • extjoblib.load() your old file as you'd planned, but then immediately re-joblib.dump() the file using the top-level joblib. (You likely want to use a distinct name, to keep the older file around, just in case.)

    移动/更新到您实际的现代环境，只有import joblib(顶级)才能使用joblib.load()-在您的代码中不再有对`sklearn.external.joblib'的引用，或者您存储的泡菜文件.

    move/update to your real, modern environment, and only import joblib (top level) to use joblib.load() - no longer having any references to `sklearn.external.joblib' in either your code, or your stored pickle files.

    这篇关于ImportError:无法从"sklearn.externals"导入名称"joblib"的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！