9.03
...
·
 |
|---|
Hello all. I’ve just finished a large photo project, 102 hours. I also just built a new computer. I’m having an issue, WBPP won’t complete a stack of my 102 hours of data. It chugs along until it gets to the luminance integration and just stops proceeding. Task manager shows pixinsight using about 6% processor and 28-98% of ram but the status never advances from about 10%. This has happened twice now, the last time I waited 5 hours for some progress and then gave up and canceled WBPP, however pixinsight hung up at that point and I had to shut it down with task manager.Here are my computer specs, Ryzen 9950, 96 gig ram, 3 fast ssds, Windows 11 pro.Any help would be appreciated. Thanks, Dan  |
0.00
...
·
·
1
like
|
|---|
Slt, en analyse, tu peux essayer de faire ton prétraitement avec Siril pour vérifier que rien ne bloque dans tes fits.
|
7.29
...
·
·
1
like
|
|---|
Maybe better to post this in the pixinsight forum directly? (Wow, 96gigz of ram 🤯) |
0.00
...
·
·
2
likes
|
|---|
You can't eat an elephant whole, but you can cut it into pieces. Break the problem into two, three parts, then put the image together from the stacks
|
11.54
...
·
·
2
likes
|
|---|
Big data is big data. You have a large dataset and you are using LN and probably the ESD rejection algorithm. You can do a back-of-the-envelope calculation if 1547 9576 x 6388 XISF files can fit to RAM (even if you think 96 GB should suffice). And on top of that there is the memory footprint of the rejection algorithm used… plus the memory footprint of Local Normalization. You can switch off LN and choose a simpler rejection algorithm (Winsorized Sigma Clipping for example), or just use FastIntegration  |
0.00
...
·
·
2
likes
|
|---|
I don't know if this is going to help. But this is my story. I was noting my processing slowing down, sometimes crashing with heavy ones (out of WBPP). So I monitored CPU usage and temperature, and it was clearly overheating. Modern CPUs automatically slow down when warming up too mcuh in order to protect its integrity. I did a deep CPU fan cleaning that only helped a little bit. Eventually I decided to bite the bullet and go for a liquid refrigeration system. Just something in the average, not too fancy. And… that made a massive difference. No more crashes, no more slowing down, CPU always at 100% clock speed, etc. I could recommend mine but I don'y if it is ok here. Drop me a PM if interested. |
 ...
·
·
1
like
|
|---|
I experienced a similar problem when stacked twice 1100 subs. I can advise you to restart the PC just before running the PI and WBPP and to avoid running anything else. Usually, I hibernate my system, but I found it ends with a much smaller amount of available RAM than I expected. I have 64GB RAM. Look at your screenshot, there is only 25(ish) GB RAM available!  |
0.00
...
·
|
|---|
Have you tried reinstalling Pixinsight? You never know…
|
0.00
...
·
·
2
likes
|
|---|
Sergey Volkov: This. I used to have similar issues with 600-700 frames already but more than 3000? I got a bit further by messing with the windows swap file settings but you never know if it will crash or not. I believe it is much easier to follow Sergeys advice. Nowadays I simply stack each night, then combine with star alignment and image integration. I think the minimum you need is 3 sets for image integration. |
10.46
...
·
·
3
likes
|
|---|
Dan Brown: Try to reduce the number of threads that PixInsight is allowed to use: EDIT -> Global Preferences -> Parallel Processing and Threads -> Maximum Number of Processors Used. Also uncheck 'Enable thread CPU affinity'. By default PI will use 32 threads on your CPU. You should set the option to 16, or even lower to 8. I have a Ryzen 5950X with 64Gb RAM and setting the value to 16 helped in my case when integrating 1000+ frames. It would be interesting to know if this will also help with your data. Vitali |
...
·
|
|---|
Vitali:Dan Brown: Is it going faster this way or just doesn't stuck? |
5.01
...
·
·
1
like
|
|---|
Lots of great suggestions here and hopefully one or a combination of several techniques will help. You didn't mention it, but you probably already monitor cpu temperature. As mentioned, CPU throttling will definately put a big halt to WBPP. I believe that Ryzen Master has that capability, but I just use the free application, Core Temp. I have an older (~5 yrs) modest Ryzen 7 system and I found that with large datasets, my system would slow almost to a crawl. I installed Core Temp to check the CPU temperature and found that I was pegging my maximum temp of 85C quickly. I am using a pretty small case and thinking that might be causing an issue, I removed the case panel and ran WBPP and found that my CPU temperature dropped >10C when WBPP is using 100% of the CPU. My system is in need of a replacement soon and as lame as sounds, I just remove the side panel when running WBPP and all is well. You probably do monitor the CPU temperature, but just in case someone else has a similar issue, it is very easy to try this out! I will be following this thread to see how things go for you. Good luck on a speedy resolution! |
0.00
...
·
·
1
like
|
|---|
I had a similar issue (with similar hardware) with my andromeda project which sits at 1500 frames over 9 nights. I had *enabled* CPU affinity, which did cause PixInsight to lockup. Going back to default settings saved the day.
|
|
0.00
...
·
|
|---|
Oh - and make sure you're running off a locally attached hard-drive and not a network connected NAS.
|
1.20
...
·
·
1
like
|
|---|
As others have said, you may be pegging your core temp too high. I have a similar system, although only a 7950X (still 16c/32t). I "only" do OSC data and "only" APS-C sized, but did a WBPP integration of somewhere near 1100 subs. It took a while, but got through it fine. You don't mention anything about your cooling, whether it's air or liquid-based. I do have an AIO cooler on mine and while it runs pretty darn warm while integrating and doing LN, it's never had an issue. Somebody asked if the whole data set can fit in memory for LN and rejection. To my knowledge, the whole data set doesn't need to fit into memory. The data gets processed and compared in chunks. Others have mentioned cutting back on the threads - FWIW, I don't cut back at all. WBPP manages the threads and RAM well. They've also mentioned the swap files. While a good idea to make sure they're set up and tweaked right, to my knowledge, they're for while you're doing post-processing and don't come into play during WBPP. I do recommend the suggestion of rebooting the PC and running nothing else before stacking a large data set. I've found that Windows can grab memory and not let it go before. I believe disk access can even be affected by Windows. A clean start definitely can't hurt… Of course, I could be wrong on any of this except for my own experience… |
|
10.46
...
·
·
1
like
|
|---|
Vitali:Dan Brown: It was faster with less threads. The CPU load was much higher during integration. Last time I had this problem with a M3 image that had 1200 light frames. I had to abort integration after a couple of days with default settings. Then I've changed the number of processors to 16 and integration has completed at a reasonable time. I do not remember exact duration. |
3.61
...
·
·
1
like
|
|---|
Hi, have you checked the log that WBPP creates as it runs through its steps? It's in a sub folder of the WBPP folder where the images are being saved. There might be expanded data there on what exactly it is hanging up on. IMO if the CPU is just sitting at 7%, it is not thermal throttling, but WBPP just gave up/crashed at some point.
|
0.00
...
·
|
|---|
Hmmm. It could be a CPU issue but on the other hand, I just ran a 2650 frames integration coming from a Nikon D780 that has a 6048 x 4024 resolution. I did this on a MacBook Air M2 having 16GB. It completed in about 18 hours in APP.
|
|
0.00
...
·
|
|---|
Hello, I am no sure, if I am right: The CPU has 32 threads, this means it would be possible to address 128GB of RAM. If the RAM is not enough, windows uses the so-called pagefile.sys to swap data, which are not used at this moment. If you will stack so many pictures, I think the available RAM if not enough for this and harddisk space is used. I am using APP to stack pictures and already stacking of about 60 pictures needs 25 GB of harddisk space. I would try to increase the allowed size of the pagefile.sys. I am using an Intel7 (8Threads) with 32 GB of RAM and I have already stacked about 200 Pictures of a Full Frame Camera (Nikon Z6). My pagefile.Sys is manged by Windows and I have set it on a SSD. However it takes a lot of time. |
3.91
...
·
|
|---|
The OP may have run out of Win11 system resources. I needed to increase Virtual Memory 3x to do large datasets. I also agree that they need to post on the PI forum. Jim Phoenix, AZ |
|
9.03
...
·
·
3
likes
|
|---|
Thank you all for the input. I will keep you posted on my progress. I'm using a liquid cooler, Corsair iCUE H150i ELITE CAPELLIX 75 CFM Liquid CPU CoolerI just downloaded software to monitor the cpu temp. Dan |
|
9.03
...
·
·
1
like
|
|---|
I just ran a small batch in WBPP and monitored my temps and task manager. The cpu is being utilized up to 100% and my ram was used up to 60%. The cpu cooler water temp kept rising for the first 10 minutes and after that my cpu temp reached 95c several times in 20 minutes. Do you think this could be an issue? Thanks, Dan |
|
5.01
...
·
|
|---|
Good possibility. I believe that 95 deg. C is the max temperature for your cpu.
|
|
3.61
...
·
·
2
likes
|
|---|
I don't think the temperature alone is the issue. Google "Ryzen 9950 95C", and the results show these chips are designed to hit that temperature and run fine. I have a 13700K, and even though benchmarks (Cinebench, 3DMark, etc.) don't get that hot, WBPP makes it ping at its limit of 100C during local normalization but it doesn't crash. I used to be concerned about the temperature, but research showed that modern CPUs are designed to hit these high temperatures without damage, they just thermally throttle unless you have an incredible liquid cooler. I am not familiar with your cooler, but if you expect better it might be worth making sure it's seated on the CPU properly. Good luck troubleshooting! -Chris |
1.51
...
·
·
5
likes
|
|---|
I've run into this issue before. It's your pixel rejection algorithm. The Generalized Extreme Studentized pixel rejection will grind to a halt if you have too many images. Use any other pixel rejection method and you'll be able to process all in one go. From the testing I've done, PixInsight will not utilize the CPU properly when using GES if the number of frames is too large. |
