The Astrobin All Sky Survey: A proposal for a community resource Other · Brian Boyle · ... · 363 · 13507 · 59

This topic contains a poll.
Would you be interested in contributing towards an AB all sky survey?
No. I wouldn't find such a survey useful.
No. Satisfactory data already exists for me elsewhere.
No. I would find such a survey useful, but I don't have the time, location or equipment to contribute.
Yes, I would be interested in taking part. One or two fields maximum.
Yes, I would be interested in taking part. Prepared to do multiple fields.
Astrogerdt 0.90
...
· 
·  Share link
I did some tests with ABE/gradient reduction in general prior to mosaic generation with data generated by Biran. I found out what we already assumed during earlier discussion, the less gradient reduction is done, the better it is. Smaller fields just don't provide enough data over actually present gradients in the image, mosaics are better or that purpose. But that was tested only with dark sky data, I have no idea wheter that is also true for higher bortle calsses with stronger gradient. 

CS Gerrit
Like
daniele.borsari 5.25
...
· 
·  Share link
With a Canon 400D and Samyang 135mm I would get a pretty widefield. The only thing is that with such an old camera the SNR is low and I should do more integration. Anyways, the idea is good and I'm in.
Like
MichaelRing 4.64
...
· 
·  1 like
·  Share link
@Daniele Borsari : Welcome to the party!!!!
Like
Astrogerdt 0.90
...
· 
·  1 like
·  Share link
Great to have you on board @Daniele Borsari !

May I ask you from which country you are, just out of interest? Currently, I have the impression we are a little low on participants from the Northern Hemisphere. 

One thing to be especially careful with, regarding your camera, is amp glow. That will require some quite food calibration to reduce the impact on the mosaic, but that should be achievable. 

CS Gerrit
Like
daniele.borsari 5.25
...
· 
·  1 like
·  Share link
Great to have you on board @Daniele Borsari !

May I ask you from which country you are, just out of interest? Currently, I have the impression we are a little low on participants from the Northern Hemisphere. 

One thing to be especially careful with, regarding your camera, is amp glow. That will require some quite food calibration to reduce the impact on the mosaic, but that should be achievable. 

CS Gerrit

Hi, I'm from Italy and I'm in Bortle 5 skies. Regarding amp glow, I've never seen it as a problem since I can get rid of it through ABE or DBE.
Here's is a shot of IC 1396 with 3h 40m of integration, I think the background is quite even.
Like
daniele.borsari 5.25
...
· 
·  Share link
Since I'm a month late, can anyone update me on the status of this project? Thank you in advice!
Also, I have a Sky Watcher Star Adventurer and I'm not able to guide/dither.
Edited ...
Like
profbriannz 17.56
Topic starter
...
· 
·  Share link
Michael Ring:
@Brian Boyle  The NoGradient Data has already had GraXterminator applied, for me the end result looks more uniform but the documentation says that this is not necessary so I may be wrong.

Did you manually select the position of the seam between two fields on my data? It helps to get smooth transitions when you make sure that there is no bright star at the seam. PM has nice tools to make the gradient between two fields as smooth as possible.

I am working on a similar complex dataset as yours that I received from Todd, hope that I can give you some hints on how to best create the pano when the data is not properly aligned. One thing I can say is that cutting your data into pieces actually helps.


Michael



Hi Michael,

No manual selection of field edges.   But I did use TrimMosaic to remove 5 pixels around the edge of each frame after using MosaicByCoordinates.  This is essential to avoid seams in the stitching process.  I did not try to manually trim field edges to avoid bright stars, as this becomes very time consuming. And I wan't to find a process that can be applied without too much manual intervention.

For large data sets, with non-uniform orientations, I agree that cutting into smaller sections may be the answer.  I will start exploring that too.

Brian
Like
Astrogerdt 0.90
...
· 
·  1 like
·  Share link
@Daniele Borsari We have a table with the fields we need for the survey: https://docs.google.com/spreadsheets/d/1eyHT4dZhSlMvfJHPtBrt93e5wDW6eJbs23bMJ7gjz1o/edit?pli=1#gid=1598116401

A drop box to store all the preprocessed fields: https://docs.google.com/spreadsheets/d/1eyHT4dZhSlMvfJHPtBrt93e5wDW6eJbs23bMJ7gjz1o/edit?pli=1#gid=1598116401

And an acquisition and preprocessing guideline: https://docs.google.com/document/d/1QZxLRpfVuxSxTDWNpjHYDpZ7p9Ua6vdz/edit#heading=h.qjx6i9yfbwrr

Not being able to dither could be a problem. We require dithered data, so that DrizzleIntegration in PI can work. This is essential for the preservation of accurate star colors, fine details and SNR. Let's see if we can figure out a solution for that. 

CS Gerrit
Like
daniele.borsari 5.25
...
· 
·  Share link
@Astrogerdt Thanks for the response. Without a modified camera I would call myself out of the game, tho for some bright broadband targets like Andromeda it could work. May I ask you to link again the drop box to store all the preprocessed fields?
Also, do I need to drizzle even if the scale is at 8.7 arcsec/px?
Edited ...
Like
Astrogerdt 0.90
...
· 
·  1 like
·  Share link
Of course, seems like I didn't copy the link in the last message: https://www.dropbox.com/scl/fo/gib8saec1dq7tt9gju66z/h?dl=0&rlkey=64tv8n9bnbk37c0g5poownt24

In most cases in our survey, drizzle is less about upscaling the images by x2, but instead about increasing the SNR of the integration by using CFA drizzle. This way, we prevent interpolation during the registration process, which leads to finer details and an increased SNR. 

An unmodified camera will lead to very visible color differences between the different panels. The filter present in normal cameras changes the color response of the camera and leads to a different white balance. Also, even around M31, there are faint H Alpha nebulas. I don't know whether we will be able to capture them with our desired SNR, but unmodified cameras will have basically no chance to capture them. 

Note that this is only my opinion. It is up to all of us to decide on this. 

CS Gerrit
Like
daniele.borsari 5.25
...
· 
·  Share link
@Astrogerdt I've always known that using drizzle decreases SNR because the noise that is on 1 pixel would be spread across 4/9/16... pixels. Regarding modded or not modded camera, let me know.
Like
Astrogerdt 0.90
...
· 
·  1 like
·  Share link
You are right, that the algorithm spreads the noise about more than one pixel which in fact increases the noise. But the noise introduced by interpolation algorithms during the alignment process of the images is much greater than that. Since CFA drizzle bypasses this interpolation, we have less noise in the end, because we add a little but subtract a much greater amount of noise. 

Additional benefits are more accurate representation of small scale structures such as stars and nebula fragments, but you probably know that. 

Note that this is true for CFA drizzle and drop shrink = 1 in PixInsight, I don't know how other software you may use handels this. 

CS Gerrit
Like
daniele.borsari 5.25
...
· 
·  1 like
·  Share link
@Astrogerdt Just did some tests on data I already had with drizzle 1x and drop shrink 1 and it turned out great, especially on star/nebula color and SNR. Nice tip I didn't know!
Comparasion between no drizzle and drizzle 1x (ABE, SPCC).
Edited ...
Like
profbriannz 17.56
Topic starter
...
· 
·  Share link
Finally managed an OK mosaic of my old Scorpius data using PI MosaicByCoordinates and PhotometricMosaic

scorpy7.jpg

Even though it works it is very tedious to do...  and as suggested by @Michael Ring  it is much easier to do in small bits. 

There is another issue that faint structures become increasingly hard to reveal in strong stellar gradients e.g. the Dark Wolf Nebula to the top right of this image.

These two reasons suggest we might want to think about producing the atlas in distinct panes (back to the original idea), using the 7.5 x 5 field centres derived by @James Tickner but merging then up to 15 x 15 deg panels  - around 200 for the whole sky.  This is only a guess - it could be 10 x 10 or 12 x 12deg.  

@James Tickner given that each RA strip is offset in its centres, I guess this means that each 15 x 15 deg region will need between 6 - 9 7.5 x 5 deg fields to complete.  But putting them together shouldn't be too onerous.   The mosaic about was 24 panels and it took me a few days... and even then I have to bin-up to 10arcsec pixels before merging.

@James Tickner I can't find your polar fields - could you let me know where you put them, and I will try to create a polar field around this size.
Like
Astrogerdt 0.90
...
· 
·  1 like
·  Share link
Wow Brian, that mosaic really looks amazing, truly great work! 

I really have no idea anymore about how to tell whether these gradient are stellar or artificial. Even when we create smaller fields, we will still have that problem, since our goal is to reveal faint nebula and providing accurate data about what is up there. So having gradients in smaller fields will be a problem too. 

CS Gerrit
Like
james.tickner 1.20
...
· 
·  Share link
Brian Boyle:
@James Tickner I can't find your polar fields - could you let me know where you put them, and I will try to create a polar field around this size.


@Brian Boyle The four polar fields are in my Dropbox account. I've shared them with you via your gmail address - hopefully you should have received a notification via email (or Dropbox). If you can't find it let me know and I'll try unsharing and resharing.

Unfortunately I wasn't able to accept the link to share your Dropbox account as my storage limit wasn't large enough - oddly, it seems you can only share an account in read/write mode if you have enough space to accommodate the entire shared account. Seems to defeat the point of sharing if you ask me!

BTW, the four fields were collected with an unmodded camera (Nikon D5600 + Samyang 135 mm F/2), so I wouldn't be surprised if you run into some colour balance issues. But it will be interesting to see the impact. I'm still working on modding my D5000, with the latest hiccup ordering some parts to make a shutter control cable to overcome the lack of USB-controlled bulb mode on the D5000. One step at a time!
Edited ...
Like
james.tickner 1.20
...
· 
·  2 likes
·  Share link
I've taken advantage of South Australia's wettest June in 35 years to work on the stitching software. To recap the ideas from some of my earlier posts, the aim is to:
  1. Accurately plate solve images
  2. Use this information to calculate the RA/DEC coordinates of each image pixel
  3. Use the RA/DEC results to reproject neighbouring images onto a consistent grid (gnomonic projection, 10x10" pixels)
  4. Compare the background and star brightness information in image overlap regions to (a) estimate and subtract any residual gradient differences between neighbouring fields and (b) estimate and correct for any overall R/G/B normalisation differences between neighbouring fields
  5. Save fields which are now fully corrected for geometry distortion, gradient differences and colour normalisation into well defined-panels (eg 15 x 15 degrees)
  6. These panels (still in linear format) can then be easily combined into larger images if required, or otherwise processed for final display


So far steps 1-3 are completed. 

For step 1, I originally hoped to use the online astrometry.net service. Unfortunately, the online version doesn't let you control which index files are used, so the solve often succeeds early using a large-scale index that results in only a few stars (10-30) being used for the solution. This is fine for determining the plate centre and orientation, but not for fitting out image distortions. After some messing around, I managed to get astrometry compiled and running on my laptop and this gives much finer grained control. Using the h4210 index (designed for ~1 deg fields) results in 400-500 stars contributing to the solution for a 6x9 field. This allows distortions to be fitted with a 3rd degree polynomial, resulting - as far as I can tell - in excellent registration accuracy across the whole field.

For steps 2 and 3, I've written some custom code in Matlab + C to perform the geometric transformations and handle the image reprojection. I've tried to keep the reprojection exact, so that flux is preserved and each original image pixel is mapped onto the new grid according to area of overlap. This seems to nicely preserve star shapes after rotation and should also accurately maintain colour calibration. The code is fairly optimised - calculation of pixel coorindates takes much less than 1 sec for a 24 megapixel original image, and the reprojection takes 15-20 secs. As everything is scripted, it should be fairly easy to automate the handling of the whole project as fields accrue.

The image below shows a sample overlay of the four polar fields that I've acquired (each about 6.5 x 10 degrees). I've converted each field to a monochrome image and then coloured them red, green, blue and yellow respectively. The aim is to test the quality of star alignment in the overlap regions - misaligned stars would generate coloured fringes when the two contributing images (red and blue, say) overlap. But if stars in the red/blue overlap are all uniformly purple then the alignment is good. I hope that explanation makes sense!

Anyway, to my eyes the star colours in the overlap regions are pretty consistent, indicating sub-pixel registration accuracy. 

Onto step 4 next!
mosaic_combined.png
Like
MichaelRing 4.64
...
· 
·  Share link
@James Tickner@Brian Boyle you two should do the sharing of files via the invite to upload files that Brian can send to you.

Sharing Dropbox is really a mess, I closed my account because I really hated that they told me that I'd need to buy a large package just to be able to collaborate with Brian.

Michael
Edited ...
Like
Astrogerdt 0.90
...
· 
·  Share link
@James Tickner Wow, amazing achievement by you, that is real dedication there! I am looking forward to the final product. 

CS Gerrit
Like
MichaelRing 4.64
...
· 
·  Share link
@James Tickner  Nice work! I think what you have achieved so far is already a real timesaver compared to manually platesolving in PI where you have to be in a quite close ballpark with estimated coordinates when trying to platesolve an image that has no coordinates.

Do you think you can appy what you have to Todd's data? it would be good to populate all 10x10 (or 6x9) tiles in our grid that are possible, the next step in processing can then likely be done in PM until you were able to do more of your magic. For me the timeconsuming part is to find the right data that makes up a 10x10 field and to then align, actually creating the pano in PM was so far not as time consuming as those steps. Are you also able to align all frames to 0° ?

Michael
Edited ...
Like
james.tickner 1.20
...
· 
·  1 like
·  Share link
Michael Ring:
Do you think you can appy what you have to Todd's data? it would be good to populate all 10x10 (or 6x9) tiles in our grid that are possible, the next step in processing can then likely be done in PM until you were able to do more of your magic. For me the time consuming part is to find the right data that makes up a 10x10 field and to then align, actually creating the pano in PM was so far not as time consuming as those steps. Are you also able to align all frames to 0° ?


@Michael Ring As my code stands at the moment, I can take one or more images with an arbitrary centre, rotation and image distortions and reproject this onto a well-defined grid with the following properties:

- User-defined centre point in RA/DEC
- User-defined image scale at centre point (currently 10 x 10" but easy to change)
- Aligned at 0 degrees (could be changed with a bit of work)
- Gnomonic projection (could be changed fairly easily)

For the purposes of forming a mosaic, it makes sense to project all of the fields onto the same grid with some suitable RA/DEC centre point (normally the centre of the overall mosaic). Alignment of these images is then trivial - you just lay them one over the other and everything lines up. At that point you can start blending, gradient adjustment, stretching etc - basically whatever work flow you want. 

Happy to have a go at Todd's data - I've PMd you separately with some questions about files.
Like
profbriannz 17.56
Topic starter
...
· 
·  Share link
@James Tickner  wonderful work on the stitching....  I found the message [in my spam...] and I am now dowloading the files.  Will see how PI gets on with stitching them together, and to my -80 dec data.
Like
profbriannz 17.56
Topic starter
...
· 
·  1 like
·  Share link
Merged 2 of @James Tickner fields at -85 [corrected using ABE to remove residual amp-glow]. with some of mine at -80 and -75 with PI's PhotometricMosaic.

pole.jpg

Result shows shortcomings PI's PM script, with does a great job with seamless matching of panels, but has the severe [crazy?] limitation of requiring othogonality.  Impossible over large areas and/or near pole.    In addition, the pair-wise approach to stitching is incredibly cumbersome and prone to error.  I suspect it will be a long wait for PI to take this code [great for sticthing] and turn it into something that is more robust and easy to use. 

It is over to you @James Tickner

Having said all that - the fields from @James Tickner and I do match pretty well with is rather encourging.
Like
profbriannz 17.56
Topic starter
...
· 
·  1 like
·  Share link
After more non-linear processing on a contiguous part of the mosaic...

Mosaic1.jpg

As predicted the use on a unmodded camera (predominantly in lower left part of the image has led to colour balance issues.  The unmodded camera images have a blue cast, which also affect some of the astro camera frames through the stitching process.
Like
james.tickner 1.20
...
· 
·  1 like
·  Share link
Accepting the issues with colour balance, that's not looking too shabby! Encouraging that panels collected using very different equipment (eg 200 mm v 135 mm lenses, DSLR v astrocamera) are stitching together so well. 

This whole project might just work smile
Like
 
Register or login to create to post a reply.