Home
Artists
Search Recent Random
Posts
Search DMs Popular Hash Lookup Tags Random
Importer
Import FAQ
Register
Partychan Telegram Find&Fuck Waifus ThePornDude
Home Artists Posts Import Register

4K 2700 Real Class Images + Auto Cropping Script (Patreon)

Published:
2023-06-04 15:18:52
Edited:
2023-08-28 09:00:00
Imported:
Tags:

Downloads

Content

Use this updated dataset for both man and woman > https://www.patreon.com/posts/87700469

Patreon exclusive posts index

Join discord and tell me your discord username to get a special rank : SECourses Discord

19 August Update

  • download_man_reg_imgs.sh file will download all of the reg images and automatically extract them into the RunPod /workspace/regimages folder. That can be used for Unix and possibly for MacOs systems as well. Don't forget to comment the links that you don't want to download and change folder paths if you wish.
  • Upload into workspace folder of RunPod and execute below command
  • cd /workspace
  • chmod +x download_man_reg_imgs.sh
  • ./download_man_reg_imgs.sh

10 July 2023 : face cropper added. it has different requirements.txt and cropper file.

The video for this post released : https://youtu.be/QTYX0tgA5ho

Please read carefully

Auto cropping script cropper.py will take your raw images and crop the subject (person) based on predefined aspect ratios with maximum efficiency. This tool is amazing to prepare training images for both classification and training.

Cropping script can be used for other objects as well such as cars. For cars here the code (line 46) : car = next((x for x in results.xyxy[0] if int(x[5]) == 2), None)

I have spent like a full day to code this script from scratch.

I am working on a new workflow to generate amazing quality realistic images. They will be beyond studio quality. For this task I needed real pictures. Therefore, I have prepared 2700 4K resolution real images for "man" class.

I have collected the images from https://unsplash.com/

They are free to use even for commercial purposes

Majority of the images had like 4000x6000 pixels original resolution.

Watch the above video to learn more about the dataset

I have manually picked the images. Images were portrait orientation. Which is the part of my new workflow. Hopefully I will make a new video where I will show my new amazing training workflow.

Then I used the attached script to extract subject into the following aspect ratios

(512, 512), (512, 768), (768, 512), (640, 960), (960, 640), (768, 1024), (1024, 768)

Then I used automatic1111 to resize them to these resolutions with focusing face. Because since the orientation was portrait, some of the images had to be cropped to be downscaled to this resolution

Below all of the images links. Each one is a zip file and the password of the zip file is:

secourses

Just plain secourses nothing else is the password

I also have uploaded the original raw images if anyone needs

These images can be used as classification / regularization images during training with DreamBooth or LoRA. They can be even used for fine-tuning training.

These images would likely to work best for realism training. For styling it may not work best. Need to be further tested

The class would be man for these images or photo of man

I use this Realistic vision (v5) model for realism (4 GB) : https://huggingface.co/SG161222/Realistic_Vision_V5.1_noVAE/resolve/main/Realistic_Vision_V5.1.safetensors

Raw images (7.54 GB) : https://huggingface.co/MonsterMMORPG/SECourses/resolve/main/raw_2735_imgs.zip

512x512 (0.91 GB) : https://huggingface.co/MonsterMMORPG/SECourses/resolve/main/512x512_2734_imgs.zip

512x768 (1.32 GB) : https://huggingface.co/MonsterMMORPG/SECourses/resolve/main/512x768_2734_imgs.zip

768x512 (1.25 GB) : https://huggingface.co/MonsterMMORPG/SECourses/resolve/main/768x512_2735_imgs.zip

768x768 (1.94 GB) : https://huggingface.co/MonsterMMORPG/SECourses/resolve/main/768x768_2734_imgs.zip

640x960 (1.99 GB) : https://huggingface.co/MonsterMMORPG/SECourses/resolve/main/640x960_2733_imgs.zip

960x640 (1.89 GB) : https://huggingface.co/MonsterMMORPG/SECourses/resolve/main/960x640_2735_imgs.zip

768x1024 (2.51 GB) : https://huggingface.co/MonsterMMORPG/SECourses/resolve/main/768x1024_2724_imgs.zip

1024x768 (2.43 GB) : https://huggingface.co/MonsterMMORPG/SECourses/resolve/main/1024x768_2734_imgs.zip

1024x1024 (3.38 GB): https://huggingface.co/MonsterMMORPG/SECourses/resolve/main/1024x1024_2734_imgs.zip

To use these files unrunpod

You need to install 7zip

  • yes | apt-get install p7zip-full

Download with

wget

e.g. :

Then use this command to extract them

  • 7z x 1024x768_2734_imgs.zip
  • or another one
  • 7z x 1024x1024_2734_imgs.zip

It will ask you password

type

  • secourses

typing will be invisible

type again and it works

How to use face cropper

Download requirements_face.txt and cropper_face.py

Set your face crop percentage

Comments

Anonymous

Thank you very much for your work and the information you've gathered. I'm impressed by all the details you've provided. I was able to train a model according to the tutorial but I still have a question. I have the impression that when I generate images I have a lot of faces that don't look like "me" sometimes they are perfect but only 1 time out of 10/15 images generated. Often the facial features are similar, but nothing more. What could be causing this, that my "selfie" dataset isn't providing enough information (about 10 images)? Is there a way to force greater precision on the use of the face? I've tried this on a dozen checkpoints, but the results are similar. and always used (subject man:1.1) in positive prompt. Thanks in advance for your answers.

Furkan Gözükara
>>114361224

thank you so much for your comment. it is 100% related to the training dataset. if you can get better dataset it will become better. so what is better dataset? different clothing and different background having images. but they have to be high quality good focus and good lightning. moreover as you do a training with higher resolution you will get better results. at least 768x1024 on realistic vision latest version i suggest

Mig Test (edited)

Comment edits

2023-08-11 16:44:46 I'm trying to crop to square only, but most of the outputs are not square. I've tried 512x512, 1x1, 1024x1024. All result in the same final size and aspect ratio. I also notice the problem files all have "Loops: 0" Final coords: (0, 21, 682, 1018), Final ratio: 0.6840521564694082, Loops: 0 The ones that do crop to square have Loops > 0 as in: Final coords: (30, 0, 710, 680), Final ratio: 1.0, Loops: 160
2023-08-08 22:26:07 I'm trying to crop to square only, but most of the outputs are not square. I've tried 512x512, 1x1, 1024x1024. All result in the same final size and aspect ratio. I also notice the problem files all have "Loops: 0" Final coords: (0, 21, 682, 1018), Final ratio: 0.6840521564694082, Loops: 0 The ones that do crop to square have Loops > 0 as in: Final coords: (30, 0, 710, 680), Final ratio: 1.0, Loops: 160

I'm trying to crop to square only, but most of the outputs are not square. I've tried 512x512, 1x1, 1024x1024. All result in the same final size and aspect ratio. I also notice the problem files all have "Loops: 0" Final coords: (0, 21, 682, 1018), Final ratio: 0.6840521564694082, Loops: 0 The ones that do crop to square have Loops > 0 as in: Final coords: (30, 0, 710, 680), Final ratio: 1.0, Loops: 160

Furkan Gözükara
>>114796169

yes some images won't be square. because this script will not crop subject. lets say you have a tall person. you can't crop it as square. you need to do another processing with automatic1111 and crop subjects. that way it will become exact resolution. if you message me on discord or if you watch video you will see.

Gireiyu

I just came here to drop a huge "good job" to you. I work as data engineer and just started having a look into the ML staff, and I know that what you have achived here is not simple and requires time. I wrote a stupid python script for face deetection using SDD_mobilenet_coco* and frozen_inference as dataset but even after playing around with the trashold value the result is always pretty bad compared to what you have done. might I ask you where can I find good staff in terms of code example and models for this kind of work ?

Furkan Gözükara
>>115773236

thank you so much. i usually as reddit and gpt 4. hopefully i will release even better 2 scripts soon. 1 uses yolo v7 and another uses pil library to focus face and downscale. i am processing woman dataset right now. hopefully i will release new man dataset too