Cosmic Clarity Satellite Removal Tool [Deep Sky] Processing techniques · Franklin Marek · ... · 32 · 1076 · 3

seti_v2 19.05
...
· 
·  3 likes
·  Share link
Satellite trails are becoming more and more of a problem in our sub exposures.  Don't throw out the exposure because of satellite trails, remove the trails from the image!!

 I am sure everyone here has examples of a satellite trail making its way to our master stack.  Our stacking programs are good, but as more and more satellite trails are in your stack, the rejection algorithms have a harder time removing them.  Let Cosmic Clarity Satellite Removal do the heavy lifting of removing the satellite trail in the sub exposure first so the stacking rejection algorithm can handle the full stack much better.

 My new program has real time monitoring of a folder such that as your telescope downloads new sub exposures it can be running the satellite trail removal on them so in the morning all your subs are ready for processing.

 Please watch the video to learn the ins and outs of the program.  Version 1 is available right now!

https://youtu.be/v4QgwMZeShU

https://www.setiastr.../cosmic-clarity

 Drive Folder for the program: here



Edited ...
Like
CWTauri 22.98
...
· 
·  6 likes
·  Share link
It appears there may be some rule breaking here.
Uncalibrated frames will have an electronic signature that the AI will not be able to reproduce in the replacement of satellite trails. 
E.g. There is a hot pixel (in a trail)  that normally would be subtracted as part of calibration. 
Wouldn't it be best to apply the removal to calibrated frames? To suggest to operate on uncalibrated data seems like it is asking for trouble and certainly breaks the best practice of removing instrumental effects/signatures before further processing (of any kind).


-adam
Like
ScottF 4.52
...
· 
·  Share link
Does it save the original unaltered files?
Like
seti_v2 19.05
Topic starter
...
· 
·  3 likes
·  Share link
Adam Block:
It appears there may be some rule breaking here.
Uncalibrated frames will have an electronic signature that the AI will not be able to reproduce in the replacement of satellite trails. 
E.g. There is a hot pixel (in a trail)  that normally would be subtracted as part of calibration. 
Wouldn't it be best to apply the removal to calibrated frames? To suggest to operate on uncalibrated data seems like it is asking for trouble and certainly breaks the best practice of removing instrumental effects/signatures before further processing (of any kind).


-adam

even within the slice almost nothing is altered except the trail, as it is trained to keep pixel values unaltered. Remember, slices that dont have a trail are completely unaltered.  If you are very concerned with hotpixel or other subtraction worries then yes perform this in batch mode on your calibrated frames
Edited ...
Like
seti_v2 19.05
Topic starter
...
· 
·  Share link
Does it save the original unaltered files?

it doesnt do anything with the original images, those stay in the folder.
Like
seti_v2 19.05
Topic starter
...
· 
·  Share link
DEBUG Version available for Windows users experiencing errors. Please use this to help me troubleshoot your system. https://drive.google.com/file/d/10JFmKVB7RPexO2UeLpEFKry78dnGjP9i/view?usp=drive_link mirror site: https://drive.google.com/file/d/1RdN-oNkWN3kWb8x_zloR23pQnWk-hTm5/view?usp=drive_link
Like
mxpwr 7.29
...
· 
·  Share link
Thanks for another great contribution. Really appreciate your scripts and tools.
Like
jarvimf12020 2.39
...
· 
·  Share link
This is somewhat tangential to this discussion thread.  Does anyone have detailed instructions for downloading and using Cosmic Clarity in the MACOS environment, especially within Pixinsight?  I have an M1 MacBook Pro.  All other PI scripts (e.g. all other Seti Astro features) work well but Cosmic Clarity is either very slow or doesn't run at all.  Should I just use the CC Suite in standalone mode?
Like
Rostokko 1.51
...
· 
·  Share link
Michael Jarvis:
This is somewhat tangential to this discussion thread.  Does anyone have detailed instructions for downloading and using Cosmic Clarity in the MACOS environment, especially within Pixinsight?  I have an M1 MacBook Pro.  All other PI scripts (e.g. all other Seti Astro features) work well but Cosmic Clarity is either very slow or doesn't run at all.  Should I just use the CC Suite in standalone mode?

In my experience (M3) CC is quite slow to start up - it does take half a minute or more for me; the startup is so slow that you may be tempted to think it's hung at times.
Once it runs, it runs reasonably fast, and it does take full advantage of the GPU.
I have seen the same behavior when running it in standalone mode, I don't think it has anything to do with the PI integration. I *guess* it's possibly related to requiring the Rosetta translation.
Once I got used to that, it's not a big deal for me; the quality of the final result typically justifies that extra "startup hesitation".

Edited: I actually think the startup slowness is due to the fact that @Franklin Marek is building the Mac binaries as what Mac classifies as "Unix Executable File"; I wouldn't be surprised if that makes Rosetta 2 skip caching, or if it forces them through a different translation path which introduces the described behavior.
Edited ...
Like
jarvimf12020 2.39
...
· 
·  Share link
Rostokko,  Thanks for your feedback.  I think you are having better luck than me.  CC-Sharpen took 10 min to complete within PI.  I just am not sure that I have done things correctly.  I am relatively new to PI (~4 months) but all other processes and scripts work with no issues.  I have all the RC-Astro plugins so this is not mission critical.  I am sure I am being boneheaded at some level.  thanks.
Like
Rostokko 1.51
...
· 
·  Share link
Michael Jarvis:
Rostokko,  Thanks for your feedback.  I think you are having better luck than me.  CC-Sharpen took 10 min to complete within PI.  I just am not sure that I have done things correctly.  I am relatively new to PI (~4 months) but all other processes and scripts work with no issues.  I have all the RC-Astro plugins so this is not mission critical.  I am sure I am being boneheaded at some level.  thanks.

How long of those 10 minutes is due to the startup behavior vs. the actual processing? After starting the processing in PI, click on the terminal window which should be visible in the dock, and check when you actually see the "Seti Astro" logo printed out. The time between PI start and when that logo is printed in the terminal window is what I am referring to as the "startup phase". Then you should be able to see the 1 or 3 channels being processed with a percentage number.
Edited ...
Like
jarvimf12020 2.39
...
· 
·  Share link
Thanks.  I will give it a try.
Like
Dfercho 0.00
...
· 
·  Share link
I haven't been able to run it successfully on MACOS at all. I am using a 2020 Intel iMac running Sequoia.  When I start the executable, it takes about 1 minute to bring up the GUI.  And then the app crashes a few seconds after I browse to the input and output folders and click 'Batch Process Input Folder'.
Like
seti_v2 19.05
Topic starter
...
· 
·  Share link
Can you share what error the terminal window is giving you, that would be helpful.
Like
jarvimf12020 2.39
...
· 
·  Share link
I think it's related to some sort of shell error on my MacBook Pro.  See this string:  to use zsh, please run `chsh -s /bin/zsh`
Like
Rostokko 1.51
...
· 
·  Share link
Michael Jarvis:
I think it's related to some sort of shell error on my MacBook Pro.  See this string:  to use zsh, please run `chsh -s /bin/zsh`

Which MacOS version are you running?
Like
jarvimf12020 2.39
...
· 
·  Share link
Sequoia 15.1.1
Like
Rostokko 1.51
...
· 
·  Share link
Michael Jarvis:
Sequoia 15.1.1

Interesting; I'm still running Sonoma and I don't see that; I wonder if it's related. But I'm not sure about how CC (or Python underneath) relies on zsh - if it does...
Like
jarvimf12020 2.39
...
· 
·  Share link
I had the same issue when I was running Sonoma and thought I would give it another go with the new OS.  No luck.  I will download everything again later today and post the terminal error messages.  Thanks again for everyone's help.
Like
seti_v2 19.05
Topic starter
...
· 
·  1 like
·  Share link
The zsh warning shouldn't be the issue.  there is something else going on.  The text from the terminal window would be a big help for sure.
Like
Dfercho 0.00
...
· 
·  Share link
Franklin Marek:
Can you share what error the terminal window is giving you, that would be helpful.

Here is the terminal output:

Seti_Satellite_abort.txt
Like
jarvimf12020 2.39
...
· 
·  Share link
this is what I get in the terminal.  PI just spins for 10 min or more.  Then I force quit.Last login: Mon Nov 25 09:52:20 on ttys000 The default interactive shell is now zsh.To update your account to use zsh, please run `chsh -s /bin/zsh`.For more details, please visit https://support.apple.com/kb/HT208050./Users/michaelf.jarvis/Documents/CosmicClaritySharpenDenoise_MACOS/setiastrocosmicclarity_denoisemac –denoise_strength 0.90 –denoise_mode full Michaels-MacBook-Pro:~ michaelf.jarvis$ /Users/michaelf.jarvis/Documents/CosmicClaritySharpenDenoise_MACOS/setiastrocosmicclarity_denoisemac –denoise_strength 0.90 –denoise_mode full Killed: 9Michaels-MacBook-Pro:~ michaelf.jarvis$
Like
Rostokko 1.51
...
· 
·  Share link
David Fercho:
Seti_Satellite_abort.txt


That's odd, the certificate of that URL seems perfectly valid: https://www.sslshopper.com/ssl-checker.html#hostname=https://download.pytorch.org/models/resnet18-f37072fd.pth
Like
Rostokko 1.51
...
· 
·  Share link
Michael Jarvis:
PI just spins for 10 min or more


Yeah, zsh is irrelevant here. It looks like the executable starts and then "sits there", as it does for me; but for me it "snaps out of it" after less than a minute, while for you apparently it never does - or it doesn't in any tolerable amount of time. Processing hasn't even started yet there.
Like
Rostokko 1.51
...
· 
·  Share link
David Fercho:
Seti_Satellite_abort.txt


That's odd, the certificate of that URL seems perfectly valid: https://www.sslshopper.com/ssl-checker.html#hostname=https://download.pytorch.org/models/resnet18-f37072fd.pth

Are you sure your network isn't blocking pytorch.org? Some companies are picky about allowing download from open source python or node.js repositories...
You could try downloading it manually in a temp folder with: 
curl -o resnet18-f37072fd.pth https://download.pytorch.org/models/resnet18-f37072fd.pth
Edited ...
Like
 
Register or login to create to post a reply.