Downloading FF threads

[Feanor]

Here is a script I built some weeks ago. Its pretty easy to use.

It will download a whole thread as a big .html file. Then you can convert it to .pdf to a better reading.

What you need:

 

* Ruby    (easy if you have a linux or mac)

* Nokogiri (ruby gem): install with “gem install nokogiri”

* some tool to convert .html to .pdf (I use wkhtmltopdf)

 

The settings:  (you can find the settings in the top of the script)

thread_id: the thread you want to download

page_limit: you can specify a limit of pages to download if you dont want all pages

download_attachments: choose if download or not all attachments (only files, the images are included in the html/pdf file)

usernames: you can filter post for just one or some users (check the examples in the script to see the syntax)

 

How to run:

$ ruby download_ff_thread.rb

 

If you have problem running the script, tell me.

• This topic was modified 11 years, 4 months ago by Feanor.

Attachments:

You must be logged in to view attached files.

[Saver0]

This is awesome!! I was thinking about making exactly this a while ago but couldn’t find the time.

Thank you so much for sharing 

Focus, Patience, Determination & Order in chaos

[simplex]

Feanor wrote:

Here is a script I built some weeks ago. Its pretty easy to use. It will download a whole thread as a big .html file. 

Wow – great idea!  Thanks a lot,

simplex

A good trader is a realist who wants to grab a chunk from the body of a trend, leaving top- and bottom-fishing to people on an ego trip. (Dr. Alexander Elder)

[Probabilita]

Thanks a lot feanor, this is a nice tool

[Anna]

Thanks for the share.
really good idea
I will try the same weekend.

[joshuayip]

this is awesome . I have been trying to download the thread manually.

I tried running this and I hit ‘read_nonblock’ : end of file reached EOFError. This is about 1300 post / 1.5 MB . May I know how do I go about it? I am trying to download a thread with more than 3000 post. Thanks in advance

 

Regards

J

• This reply was modified 11 years, 4 months ago by joshuayip.

[smallcat]

Feanor wrote:

Here is a script I built some weeks ago. Its pretty easy to use. It will download a whole thread as a big .html file. Then you can convert it to .pdf to a better reading. What you need: * Ruby (easy if you have a linux or mac) * Nokogiri (ruby gem): install with “gem install nokogiri” * some tool to convert .html to .pdf (I use wkhtmltopdf) The settings: (you can find the settings in the top of the script) thread_id: the thread you want to download page_limit: you can specify a limit of pages to download if you dont want all pages download_attachments: choose if download or not all attachments (only files, the images are included in the html/pdf file) usernames: you can filter post for just one or some users (check the examples in the script to see the syntax) How to run: $ ruby download_ff_thread.rb If you have problem running the script, tell me.

Thanks a lot Feanor. I tried it in windows, and it works great. 
The only thing to note is, if the thread is in “Recycle Bin” of FF (ex: similarity thread), we can not download the attachments automatically, so i set :  download_attachments = false   at  your ruby file.

In order to work properly, i need to install some other package too (ex: mini_portile), this is what i did  :

1) download & install : rubyinstaller-2.1.5.exe
2) download : mini_portile-0.6.2.gem
from ruby command prompt i did :  gem install  mini_portile-0.6.2.gem
3) nokogiri-1.6.5-x86-mingw32.gem
from ruby command prompt i did :  gem install nokogiri-1.6.5-x86-mingw32.gem
4) unzip your ruby code :  download_ff_thread.zip
edit ruby file, set  download_attachments = false 
if  we download the threads that are in “recycle bin” of FF, the attachment can not be downloaded automatically
just leave this setting to true, if the thread we are downloading is not at “recycle bin” of FF.
5) download & install :  wget-1.11.4-1-setup.exe
6) Download & install : wkhtmltox-0.12.2.1_msvc2013-win32
7) Run your ruby script from ruby command prompt :
ruby download_ff_thread.rb

It will download the whole thread just in 1 html file, this is great !
So, it is easier to convert it to Pdf 

 

[ForexJeanie]

I asked FF for such a feature via print or something else. They told me they will check for future versions – bla bla bla. They will never do because of advertising when you are online and you won´t once you can download something in which way ever.

So, it is a great idea to export FF-threads. But I am struggling with the instructions here.

I am new to ruby. I get syntax errors when installing the gems (tInteger and missing < and so on). I could not find out how to overcome these issues. Help is needed.

I located the gem-sources in the folder where other gem-files are stored. I can start interactive ruby. How can I check if a gem is correct installed when there is no error message. How does ruby tell you that a gem is successfully installed?

FJ

• This reply was modified 11 years ago by ForexJeanie.

[ForexJeanie]

Sorry, but I need some help here. Could you please elaborate in more detail what you did. I don´t know ruby and I have no idea how I can get this to work.

I installed ruby. I installed wget an the html to pdf tool. I can not install the gem-files.

Where should the gem-files be stored before you install them? Is there a special directory in ruby?

 

FJ

[joshuayip]

u just need to type the command in the the command prompt to install it.

[Author]

Posts