我可以在Python 3上提供指向lxml.etree.parse的URL吗?
问题内容:
该文档说我可以:
lxml可以从本地文件,HTTP URL或FTP URL进行解析。它还会自动检测并读取gzip压缩的XML文件(.gz)。
(来自http://lxml.de/parsing.html “解析器”下)
但是快速实验似乎暗示了其他方面:
Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:45:13) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from lxml import etree
>>> parser = etree.HTMLParser()
>>> from urllib.request import urlopen
>>> with urlopen('https://pypi.python.org/simple') as f:
... tree = etree.parse(f, parser)
...
>>> tree2 = etree.parse('https://pypi.python.org/simple', parser)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "lxml.etree.pyx", line 3299, in lxml.etree.parse (src\lxml\lxml.etree.c:72655)
File "parser.pxi", line 1791, in lxml.etree._parseDocument (src\lxml\lxml.etree.c:106263)
File "parser.pxi", line 1817, in lxml.etree._parseDocumentFromURL (src\lxml\lxml.etree.c:106564)
File "parser.pxi", line 1721, in lxml.etree._parseDocFromFile (src\lxml\lxml.etree.c:105561)
File "parser.pxi", line 1122, in lxml.etree._BaseParser._parseDocFromFile (src\lxml\lxml.etree.c:100456)
File "parser.pxi", line 580, in lxml.etree._ParserContext._handleParseResultDoc (src\lxml\lxml.etree.c:94543)
File "parser.pxi", line 690, in lxml.etree._handleParseResult (src\lxml\lxml.etree.c:96003)
File "parser.pxi", line 618, in lxml.etree._raiseParseError (src\lxml\lxml.etree.c:95015)
OSError: Error reading file 'https://pypi.python.org/simple': failed to load external entity "https://pypi.python.org/simple"
>>>
我可以使用urlopen方法,但是文档似乎暗示通过URL更好。另外,如果文档不准确,尤其是当我开始需要做更复杂的事情时,我会担心依赖lxml。
从已知URL用lxml解析HTML的正确方法是什么?我应该在哪里查看该文件?
更新 :如果我使用http
URL而不是一个URL,则会出现相同的错误https
。
问题答案:
问题是lxml不支持HTTPS
URL,并且http://pypi.python.org/simple重定向到HTTPS版本。
因此,对于任何安全的网站,您需要自己阅读网址:
from lxml import etree
from urllib.request import urlopen
parser = etree.HTMLParser()
with urlopen('https://pypi.python.org/simple') as f:
tree = etree.parse(f, parser)