AI judging of IOTD, TPN etc AstroBin Platform open discussions community forum · Leela.Astro.Imaging · ... · 39 · 1160 · 3

Leela.Astro.Imaging 1.51
...
· 
·  2 likes
·  Share link
Hello,

Its raining today and while catching up w a backlog of processing and seeing what's top of the forum updates, I noticed another batch of discussions about "why is this not IOTD" or "backyard vs hosting" etc.  It is NOT my intention to open anything like that (nothing would be gained tbh from rehashing), but while watching the rain, a thought suddenly struck me.

There is by now a HUGE data set of images that made TPN, IOTD etc and those that did not.  So it ought to be possible to train some algorithms which can look at an image and determine what grade it would make!

Some might argue that removes the human element  (aka subjectivity).  But given that the data-sets the AI would be being trained on actually incorporate that prior human assessment, thus so would the trained model - and indeed perhaps a model would apply its rubrics more systematically.

I have no idea who submitters, reviewers, judges etc are but tbh there seems no need to have those folks involved any more (especially as any human review process is frankly a bit of a black box anyway if there are no feedback loops built in).

In fact, a sophisticated AI could even give feedback on images (either on an image in its own right, or a comparative score of other images of the same target).

For the avoidance of doubt, this not coming from any place of angst from me about the IOTD process (my images and my skies are nowhere near good enough to stand a chance :laughing-1smile.

But I do think that there is enough data now available to try this as an approach.  If nothing else it would be interesting to compare what an algorithm comes back with, against where the human assessment goes.  Indeed, the astrobin datasets are probably large enough to have deep historical cross-validation sets where the grading of such algorithms could be compared to what the human gradings turned out to be for those images.

Are there any AI engineers in the astrobin community who would/could try building something like this?

Cheers
Like
siovene
...
· 
·  40 likes
·  Share link
Just in case, before anyone even thinks of it, while this sets up a very interesting technical discussion, I have no intention whatsoever of making the IOTD/TP selection be selected by AI.
Edited ...
Like
andreatax 9.89
...
· 
·  9 likes
·  Share link
Salvatore Iovene:
Just in case, before anyone even thinks of it, while this is sets up a very interesting technical discussion, I have no intention whatsoever of making the IOTD/TP selection be selected by AI.

Thank goodness!
Like
jeffbax 16.19
...
· 
·  8 likes
·  Share link
😆. IA is based on data analysis. Based on the past awards, it might train not to award new visions. Astrophotography is about human perception of the universe IMO.

JF
Edited ...
Like
kevinldixon 4.62
...
· 
·  7 likes
·  Share link
Salvatore Iovene:
Just in case, before anyone even thinks of it, while this is sets up a very interesting technical discussion, I have no intention whatsoever of making the IOTD/TP selection be selected by AI.

I am so very pleased to hear that Salvatore!!
Like
alexgov 4.82
...
· 
·  7 likes
·  Share link
I think AI as a judging panel would be more appropriate for evaluating images taken by AI-controlled telescopes and processed entirely by AI smile

CS, Oleksiy
Edited ...
Like
Overcast_Observatory 19.90
...
· 
·  11 likes
·  Share link
Oh lord, im going to grab my telescope and a rifle and head for the hills. Im not made for this brave new world….
Like
HegAstro 14.24
...
· 
·  1 like
·  Share link
Maybe as a custom contest. 

You could have one contest for each AI of someone's choosing and we could then have debates about which AI was best. I expect it will be every bit as interesting as the current IOTD debates.
Like
Die_Launische_Diva 11.54
...
· 
·  9 likes
·  Share link
I can't wait to see AI arguing about whether it is considered 'cheating' when someone uses a human somewhere in the processing pipeline! smile
Like
EricGagne 3.82
...
· 
·  1 like
·  Share link
Respectfully to the OP.  When AI can create realistic Astro images maybe we can think of having it judge what we do.

Until then thanks but no thanks.
Like
Christophorus 12.36
...
· 
·  9 likes
·  Share link
If this happens one day the watching members of astrobin aren't necessary too anymore. So the AI can be used for everything. Producing,processing, judging ,watching and discussing the astrofotos. We can have a beer in that time.

Cheers, Christoph smile
Like
Leela.Astro.Imaging 1.51
Topic starter
...
· 
·  4 likes
·  Share link
I am such a luddite that I had to learn how to join this group to post a reply to my own topic  I'm enjoying seeing the answers - and am relieved to see many fellow luddites out there.

For completeness, the geek in me is ofc obliged to say that using AI-based approaches to rank and score images is nothing to do with whether AI can create images well or not.  Computer vision algorithms using ML-based scoring techniques are used in so many industrial (and even medical imaging) processes to analyse and assess pictures.  And even then there should ofc always be a skilled human at key stages in the chain.

Btw another non-AI approach which is increasingly being used in education for marking subjective work (like essays) is "comparative assessment" - based on insights from psychology from a century ago!

Studies have shown that when graders try and quantify the score of an essay, the marks are not really reliable or consistent.  But instead if graders are shown multiple sets of two randomly selected pieces of work and just asked to rank them vs each other, then when enough such ranking pairs are presented to enough graders, you can actually get a ranked curve of all the essays which shows much more robustness.  Bizarre I know but apparently the case.

So maybe instead of an AI-based engine, a comparative methodology could be used.  Batches of [N] submitted images are turned into lots of random pairs which are presented to reviewers, and instead of assessing whether a given image scores "good enough", all they have to do is click which they think is the better of that pair.  Once that has happened enough times with enough random pairs by enough reviewers, then a curve is automatically generated of those N images and top X% are moved through to the next stage, where such pair comparison happens for the next whittling down to the next stage.

Maybe that could take some of the heat out of these topics, and also make life a lot easier for the reviewers/judges b/c all they are doing is comparing multiple lots of two images randomly presented to them from a given assessment set.

Anyway, my processing laptop has finally finished crunching some data so I'm off with a non-AI beer to look at what the maths and algorithms of calibrating and stacking have revealed
Like
messierman3000 7.22
...
· 
·  2 likes
·  Share link
I think AI, even at this time, lacks the intelligence and would make a bunch of mistakes and might even turn this into an IOTD:

output.jpg
(AI image, I hope y'all recognize the fake look!)

You want people to stop complaining about IOTD? Remove it.
Like
AstroRBA 4.98
...
· 
·  2 likes
·  Share link
Die Launische Diva:
I can't wait to see AI arguing about whether it is considered 'cheating' when someone uses a human somewhere in the processing pipeline! 

Ha Ha !
Like
HegAstro 14.24
...
· 
·  2 likes
·  Share link
Oscar:
I think AI, even at this time, lacks the intelligence and would make a bunch of mistakes and might even turn this into an IOTD:


Honestly, that looks more like a dessert or cappuccino than a galaxy.
Like
messierman3000 7.22
...
· 
·  Share link
Arun H:
Oscar:
I think AI, even at this time, lacks the intelligence and would make a bunch of mistakes and might even turn this into an IOTD:


Honestly, that looks more like a dessert or cappuccino than a galaxy.

AI wouldn't know the difference! 
Like
AstroRBA 4.98
...
· 
·  Share link
Christoph Lichtblau:
We can have a beer in that time.


Bring on the AI !
Like
afd33 9.38
...
· 
·  4 likes
·  Share link
Oscar:
You want people to stop complaining about IOTD? Remove it.

How about, the people complaining just realize it's a completely optional thing that is opt in to enter. Don't like it. Don't do it. Ta-da! I fixed it.
Like
messierman3000 7.22
...
· 
·  3 likes
·  Share link
Arun H:
Oscar:
I think AI, even at this time, lacks the intelligence and would make a bunch of mistakes and might even turn this into an IOTD:


Honestly, that looks more like a dessert or cappuccino than a galaxy.

Since you mentioned about cappuccino, I decided to process a cappuccino:

hhhhhhhh.jpg
  • binary mask on the liquid
  • Invert
  • BXT correct only
  • LHE
  • Small saturation changes


surprised at how well it turned out

imagine this being an IOTD, lol
Like
messierman3000 7.22
...
· 
·  Share link
Quinn Groessl:
Oscar:
You want people to stop complaining about IOTD? Remove it.

How about, the people complaining just realize it's a completely optional thing that is opt in to enter. Don't like it. Don't do it. Ta-da! I fixed it.

wont fix anything

the people who quit will get jealous at the people getting IOTD, because the IOTD winner is getting so much attention, and then will go back in, and then will complain again because he he didn't get anything

and then someone like you will tell him: "you don't like it, don't do it!"

he will leave participation again, tell his friends and his friends will come here and complain.

those will stop, will give up; then the viewers who saw the thread of their complaints will then get ideas of their own against IOTD, and make their own threads.

It multiplies. What I said is a variation of what can happen, but it's like this.

this is what I think anyway.
Edited ...
Like
maze 1.51
...
· 
·  Share link
Salvatore Iovene:
Just in case, before anyone even thinks of it, while this sets up a very interesting technical discussion, I have no intention whatsoever of making the IOTD/TP selection be selected by AI.

Good to hear.
Like
AstroRBA 4.98
...
· 
·  1 like
·  Share link
Oscar:
AI wouldn't know the difference!


AI might add walnuts ...
Like
DarkSky7 3.81
...
· 
·  1 like
·  Share link
Oscar:
I think AI, even at this time, lacks the intelligence and would make a bunch of mistakes and might even turn this into an IOTD:

output.jpg
(AI image, I hope y'all recognize the fake look!)

You want people to stop complaining about IOTD? Remove it.

*Looks like my M94! (Yeah, no, I wish
Like
patrice_so 7.87
...
· 
·  2 likes
·  Share link
Hi there,

If the IOTD/TP system were replaced by an AI, then the community as a whole could be replaced. The AI could participate in the forum and comment on images. It could produce much more content than users actually do. But wait — other social networks are already deteriorating quite fast because they are being populated by more AI content than human content.

Astrobin is a social network. It brings people together to share a common passion. It's one of the best online communities I know. I am interested in feedback from the community, not from AI, which is mostly an echo chamber. Besides, I do not entertain social relations with an AI. 

Complaints about the IOTD/TP system are commonplace. I tend to view these more as an expression of frustration from people who invest time and resources in a challenging hobby and expect meritocratic rewards. In an age where rewards are based on likes and online visibility, it is very easy to get caught up in it. However, one should not be confused: the IOTD/TP system is working well overall, and I am glad that the people involved take the time to contribute to the community in this way.

Cheers,

Patrice
Like
bdm201170 8.64
...
· 
·  2 likes
·  Share link
Hello everyone

I have no intention of opening a Pandora's box

In general, I don't agree with using AI to select the best image of the day. However, I don't completely disagree. There are advantages. We always use disadvantages as a shield or as a way of saying no. But now let's look at the positive side and see if it's possible. And here are some advantages.

AI
AI has no personal preferences.
It has no biases or biased influence. On this topic, there are several articles from Harvard University that prove this.
AI is not affected by external agents or emotional states when choosing (another article from Yale University).
AI has no jealousy, envy, or emotions, and it will always select the best option. in % 

For example, if I teach the AI ​​to analyze an image with the percentage of “panda eye” and give me a result in percentage of each image, it will choose the one with the lowest percentage and I will add a level of importance to it in a list of totals.

the same with, over-sharpening, noisy or grainy, artificially smooth, saturation or color banding artifacts and more 

Whether or not it's functional in IOTD selection is up for debate.

CS
Brian
Like
 
This topic was closed by a moderator.