[2.80] Lily Surface Scraper: Import material from a simple URL

Download LAST RELEASE on GitHub.

paste-url

Hello, I release a script to import materials from online libraries like CC0Textures and cgbookcase by just copy-pasting the URL. That way, one can just browse the website, then quickly import the material.

You can find more information and the download link on the GitHub page, but I need your help for two things:

  • Finding more sources of textures. I am open to suggestions, and I made it as easy as possible for one to implement a new scrapper, see the README.

  • Telling me the proper way of including third party modules in a Blender add-on. This add-on relies on lxml, so I just copied it in my repository, but it is not clean at all, and I will have to ask linux users to send me their version of lxml so that I can add it to the repo. Anybody knows about this?

Hope you’ll enjoy it!

[edit]
Supported sources so far

Materials:

Worlds:

Lights:

27 Likes

Thank you this is very useful and fast.

1 Like

I just added support for TextureHaven, hence a v1.0.1:

1 Like

uhh, I’ll try this one out tomorrow. Looks super nice

1 Like

got error

Traceback (most recent call last):
  File "/home/weybec02/Downloads/blender-2.80-337cac760ba9-linux-glibc224-x86_64/2.80/scripts/modules/addon_utils.py", line 351, in enable
    mod = __import__(module_name)
  File "/home/weybec02/.config/blender/2.80/scripts/addons/LilySurfaceScrapper/__init__.py", line 36, in <module>
    from . import frontend
  File "/home/weybec02/.config/blender/2.80/scripts/addons/LilySurfaceScrapper/frontend.py", line 26, in <module>
    from .CyclesMaterialData import CyclesMaterialData
  File "/home/weybec02/.config/blender/2.80/scripts/addons/LilySurfaceScrapper/CyclesMaterialData.py", line 27, in <module>
    from .MaterialData import MaterialData
  File "/home/weybec02/.config/blender/2.80/scripts/addons/LilySurfaceScrapper/MaterialData.py", line 26, in <module>
    from .Scrappers.AbstractScrapper import AbstractScrapper
  File "/home/weybec02/.config/blender/2.80/scripts/addons/LilySurfaceScrapper/Scrappers/AbstractScrapper.py", line 30, in <module>
    from lxml import etree
ImportError: cannot import name 'etree' from 'lxml' (/home/weybec02/.config/blender/2.80/scripts/addons/LilySurfaceScrapper/site-packages/lxml/__init__.py)


2 Likes

@draguu This was expected for linux users, but hopefully can be easily solved.

The problem is that I don’t have a linux at hand at the moment. Can you install lxml on your system (or in a python virtual env) using pip3.7 install lxml, then browse to /usr/lib/python3.7/site-packages and copy the lxml directory? I will add it to the repository so that it works for everybody out of the box.

i did pip instal lxml, and copied lxml to blender2.8 python lib but it didnt work

Yes, because it still tries to load the lxml provided with the addon. Can you zip me the lxml lib so that I can embed it? And did you check that you downloaded the python3.7 version? It has to match Blender’s Python, which is 3.7.

1 Like

@draguu I finally put a hand on the linux binaries for lxml, so the new version the Lily Surface Scrapper now works on linux!

This new release also introduces the import of Worlds from HDRI Haven.

still got some errors

Traceback (most recent call last):
  File "/home/weybec02/Downloads/blender-2.80-337cac760ba9-linux-glibc224-x86_64/2.80/scripts/modules/addon_utils.py", line 351, in enable
    mod = __import__(module_name)
  File "/home/weybec02/.config/blender/2.80/scripts/addons/LilySurfaceScrapper/__init__.py", line 36, in <module>
    from . import frontend
  File "/home/weybec02/.config/blender/2.80/scripts/addons/LilySurfaceScrapper/frontend.py", line 26, in <module>
    from .CyclesMaterialData import CyclesMaterialData
  File "/home/weybec02/.config/blender/2.80/scripts/addons/LilySurfaceScrapper/CyclesMaterialData.py", line 26, in <module>
    from .MaterialData import MaterialData
  File "/home/weybec02/.config/blender/2.80/scripts/addons/LilySurfaceScrapper/MaterialData.py", line 24, in <module>
    from .ScrappersManager import ScrappersManager
  File "/home/weybec02/.config/blender/2.80/scripts/addons/LilySurfaceScrapper/ScrappersManager.py", line 26, in <module>
    from .Scrappers.AbstractScrapper import AbstractScrapper
  File "/home/weybec02/.config/blender/2.80/scripts/addons/LilySurfaceScrapper/Scrappers/AbstractScrapper.py", line 30, in <module>
    from lxml import etree
ImportError: cannot import name 'etree' from 'lxml' (/home/weybec02/Downloads/blender-2.80-337cac760ba9-linux-glibc224-x86_64/2.80/python/lib/python3.7/lxml/__init__.py)


The lxml directory in 2.80/python/lib/python3.7 is the one you copied manually, right? You should remove it since it shadows the lxml of the add-on. I suspect it is for a slightly different version of python (maybe 3.6).


I am using Linux too, and there isn’t a lxml directory in 2.80/python/lib/python3.7,the add-on doesn’t work,I don’t know why

Nice, love this idea. Where does it save everything? I can’t test it out at the moment.

@draguu @loveordie My bad, sorry, I messed up with the release zip, I just updated it:
https://github.com/eliemichel/LilySurfaceScrapper/releases/download/v1.1.0/LilySurfaceScrapper-v1.1.0.zip

@RealityFox It creates a directory LilySurface/ next to the blend file, and sort downloaded files per source site, then material name, then variant (1K, 2K, etc.). If you have several blend in the same folder, they share this directory and hence the add-on does not download again stuff that is already here. If needed I could add a way to customize this directory.

Dope, when I get home I’ll be sure to give this a go. Especially like how it works with hdri haven.

1 Like

It’s great! It works now!

1 Like

The next release of BATS (Substance Painter inspired add-on) by JoshuaKnauber will use Lily Surface Scrapper if it is installed, as shown on the demo video:

This explain the last two commits, making it easier for third party developpers to include Lily Surface Scrapper into their pipelines!

1 Like

super awesome … it works well now :slight_smile:

1 Like

It would be possible to save the top URLs in a dropdow field. not to be looking for a URL every time you need a new material. First you would search the general URL field for easier access to the material-specific URL.

Mmh why not using the list of materials already present in the scene? Once a material has been downloaded, it is accessible in the drop down list as any material.

1 Like