Hey guys, First of all thanks for python-boilerpipe Trying to use Boilerpipe but can't extract properly some documents... > from boilerpipe.extract import Extractor > extractorType="DefaultExtractor" > sourceUrl = 'http://www.indiatimes.com/news/india/arvind-kejriwal-to-seek-political-sanyas-127620.html' > extractor = Extractor(extractor=extractorType, url=sourceUrl) > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/Library/Python/2.7/site-packages/boilerpipe/extract/**init**.py", line 41, in **init** > self.data = unicode(self.data, encoding) > UnicodeDecodeError: 'utf8' codec can't decode byte 0x91 in position 53647: invalid start byte The document seems to be having some non-utf8 characters... which do not seem to parse well... Any workaround for the problem?
Activity
Caimany commentedon Jun 1, 2015
I solved UnicodeDecodeError ,you can see what I modified in init.py
https://github.com/Caimany/python-boilerpipe/blob/master/src/boilerpipe/extract/__init__.py