H. Maria - Overwhelming - suggestion


#1

I had to vent a bit about the extent of damage we see.
Now that I did, I have a suggestion for improvement.

if we can filter out polygons(tiles) that were covered by by n number of nodders that reached a certain consensus value it will allow to scan faster through a map.

Also if there is a way to classify tiles based on whether or not contain data (vote clouds out) so they can be filtered out it will also expedite the search.

below is my venting.you can skip it if you want.


The Dominica map is simply too much. I am on the Roseau area for more than an hour on just 10 tiles.
There is simply way too much damage :frowning:(https://www.tomnod.com/campaign/campaign_2128/map/1mgcx4y1g)
just needed to share a little.


#2

Agree, maybe a major damage tag, for the tiles with too much to tag.


#3

Sadly, the “Major Damage” tag probably doesn’t give the relevant authorities the detailed information they are asking for… as per Mel’s post here http://forum.tomnod.com/t/h-harvey-whats-our-impact/4793/7

I have to agree it would make things a LOT easier to have a major damage tag, I’ve also spent a long while on a very small number of tiles in Dominica (not sure where, more concentrating on putting all the tags in).


#4

I do understand “overwhelming”. I’ve been doing 1 or 2 rows, then taking a break. I switch to Harvey or Irma, and return to Maria later.

The tag campaigns are not the same as poly campaigns. It takes extra manpower to poly it.

Melissa just commented that there are about 1.5 staff working on the Tomnod side. Used to just be 1 person (Happy_Mapper) with Jon when needed. DG is a small company, so they do one heck of a lot with few in-house people. Please understand. They aren’t trying to make it harder on us. If we can just do what we can with what we have in front of us…

Helen is correct that the “Major” tag would not tell numbers involved.

We will ALWAYS have to deal with clouds, especially with storms. They haven’t figured out ways to remove them completely.

But I do understand. :frowning:


#5

I have found many small houses far away from others with no road leading to them - just a slightly worn “track” as they call them. But the tracks aren’t always visible. To make things harder, these small homes are built under the forest canopy so they aren’t always visible in the “Before” photos. Even in the “After” photos they are hard to see, especially if they are dark brown (many are). When I tag these homes, many times I am the first (and maybe the only one?), and sometimes there is only one other tag on it. If we “skip” these homes, then the total damage count is less than the true number. I simply take my time and plug along, switching back and forth between the before and after photos many times on one map tile. I am not in a race, but want to tag all damage. Not that it matters to me, but my count on the new set of Maria maps is 206 only because a lot were water, clouds or forested areas without any homes. Those add up quickly. Just plug along. We’ll get there! :slightly_smiling_face:
To give my eyes a break, I switch to the Weddell Seal campaign for 50-100 polygons. Then I switch back - still seeing white! :rofl:


#6

When I close my eyes there is a sea of pink circles.
Also, when I see footage on TV, my first instinct is to tag it.
But I do feel like what we’re doing helps. I hope the US can step it up helping Americans in Puerto Rico.


#7

As I read this, I started laughing out loud! My DIL just looked at me… waiting for me to tell her what the joke was. I took “the Fifth.” For those not familiar with this, in a nut shell the “Fifth” refers to the Fifth Amendment to the U.S. Constitution which states that one can refuse to answer a question so as to not incriminate oneself." I wasn’t about to sit and explain the countless hours I spend nodding AND the countless hours I spend reading and writing these nutty one-liners. :zipper_mouth_face::roll_eyes::grinning:

And I sincerely hope the U.S. will step in and help every nation that was devastated by these hurricanes.


#8

Oh Jim! I thought the Fifth came in a bottle? flask? LOL

According to “someone in power” via Twitter today, PR doesn’t deserve rescue or help.

Don’t want to begin any political debate here. It is just inconceivable to me that 3 & 1/2 million people are left suffering while a Leader (fill in the well-known name) thinks PR was in terrible shape before, and that they aren’t doing enough to help themselves now!! He compared PR to Texas and Florida, which, he said, are doing well after the hurricanes-- really?!? Hope the S.Service abandon him in the middle of PR with no food or water or phone…cut off from anywhere by rushing dirty, undrinkable water… grr … hiss… claws out… Cough… uhm, I’m just talking about a… uhm,… a huge squeaky mouse , not anyone in politics. Cough. :cat2: clawing :mouse2: :


#9

:mouse: :snake: :roll_eyes: :face_with_raised_eyebrow: :thinking: :no_mouth: :expressionless::neutral_face: :grinning: :rofl: :innocent:


#10

OK. I want to further this discussion.
it is related to what other wrote on other posts.

I agree. but the question is what type of data is being collected.

Just location of the damage, is good enough. damage density maps will also be good for prioritization purposes.
But counting damaged houses is more complex, as people said that they tag also pools, sheds and other installations that are not houses. So we are not counting just houses and we bias the numbers. I am not sure how much, but we do.
also not everyone is completing his/her map so the average number of tags per map is not a good numerical. I assume that the number of tags having a large census score is a more accurate number, but I had cases where I went into cloud areas or the edges of a map and was the only one that tagged a damage. So even items that are clearly damaged are not always getting high census score. So I assume there are many issues involved with the data that we generate but it is probably much better than not doing anything, but it would be great to understand better how the data is processed to allow us to create better tagging guidelines.
I think that a major damage tag is not always the answer (depending on what type of data Tn is after).

But I do think that if we can have just another button saying “there is no data in this tile” (cloud/sea/blank forest without houses/ only undamaged houses…) than it would allow to prioritize tiles. later if resources permit it one might think on a way to allow exclusion (not display or auto-skip or color differently) of tiles that reached an agreeable census score for not having data in it.


#11

Yes, that happens a lot with me too, I don’t even try to explain because it’s usually just impossible, you have to have been there to find the funny side :smile:


#12

That’s sad :disappointed:
A sad and sick attitude to have over humanity :cry:


#13

Once again… that isn’t how they collect the data.


#14

Hi all and @shahar. One thing to keep in mind is that all tags do not make it into the final dataset. We have a very good QA algorithm that we run across all of our data, which clusters and assess “reliability.”

For example, in our Harvey campaign, 534,240 tags were placed, but we clustered and assessed reliability to only 19,554 in the final dataset.

I realize it’s difficult for non-analysts to look at the data in the format we provide, and that’s something we’re looking into. But if, for example, you were to examine the Irma data, you would see that the final data set is quite accurate - tags on pools, boats, etc have all been cleaned by the algorithm. Which doesn’t imply you should just go wild with your tagging, because the algorithm relies on certain “reliable” voters to make the call.

The algorithm is also how tomnod prioritizes tiles that are viewed. Areas where the crowd is in “disagreement” get prioritized, along with areas that have not been searched. There are numerous issues why we can’t add a “no data” button - primarily because it isn’t strictly needed per the above to produce a great dataset, and because it’s not something tomnod recognizes right now, and would require significant developer time that we unfortunately, just don’t have.

If you want more info on our QA algorithm, this page is good read: http://admin.tomnod.com/nod/admin/analytics_extract_explained.html

Thank you, volunteers!


#15

I don’t think Mel will mind if I comment not to second-guess your tagging based on what clusters the computer decides on in the end. Usually our first instinct is good. As well, over time, we all learn and naturally begin to alter what we tag based on individual growing experience. The computer brings together all the individual learning, clusters it, and feeds that group effort back to the expert communities.

In the same vein, the computer “overlooks” our idiosyncrasies because it doesn’t cluster with other taggers. Like, I tag about every 1 inch along a flooded road because I cannot stand the ambiguity of placing only a begin and end tag. (Like how will the computer know the road meanders?!!) But I already know that DG computer will see my tags as overkill and discard some-- many – oh okay, most of them. But with the next flooded road, I still trace its curves through landscape as long as the blockage occurs. You’d think I’d “learn”, huh? LOL


#16

Great points, @cageycat.

Your road tagging is a decent strategy for useful final data, even though many tags might get tossed. It’s difficult with tomnod tagging campaigns because tomnod relies on points, and roads are lines. So your and other’s methods like this helps us get nice series of dots spaced along flooded roads.


#17

Especially when there are secondary roads (flooded) that branch off the main roads.


#18

Hi @Mel_Nod,
I read the explanations about the algorithm and I think I understand where the discrepancy in the conversation is.

There are two different sets of “goals and points of view” here. Tomnod’s and the user’s.
You are right, from TN point of view the mission will be accomplished anyway. The algorithm sends new nodders to priority tiles. Each nodder will be placed on a different tiles than the others depending on the tiles priority matrix set at that time point by Crowd rank (CR).
It is an effective way to search the entire map. I use similar techniques on my technical data sets to search for optimal values in my system.

But there is also another point of view, that of the user (nodder).
I rarely find myself jumping from tile to tile by the “jump to priorety area” button. too often there is an object to tag on the edge of the adjacent tile and so I move to that tile next. and to the next one. I might follow a road if I fancy tagging road blocks or just consistently walk to cover the damage in a multi-tile area. In areas I do not find anything immediately like in Houston, I do a “1 on 1 off” sampling of the map as it is quicker than moving to every tile until I start seeing something to tag and then will resume visiting adjacent tiles. only when I get frustrated I will zoom out and start fresh in a new point, or use the “jump” button.

I do that because I try to optimize my time expenditure. I am selfish in that respect. I choose to spend my time and effort (“my resources”) and I do that willingly but I try to optimize my spending of that resource for selfish reasons. I have a slightly slow (though new) computer :computer: so the tiles do not load instantly. Waiting for an empty tile to load is a waste I would like to avoid.

I am sure that if TN will look into “tile visiting patterns” it will find some interesting stuff related to human behavior and psychology. I wonder how many employ similar techniques in effort of optimizing their time. I know some are more patient than me and just go row by row or column-wise to cover the entire map, but I rarely do so.

But the point is this, regardless of the the nodder’s search pattern he or she would probably like to skip those tiles that contain nothing. Time is a personal resource (mine is spent very late at night or early morning [4am] ) that we choose to spend at tomnod not because we have nothing else important to do but because we believe we are doing an important thing and helping someone in need. That time is a precious commodity, and the reason why the user need to be considered too. We will continue to “grit teeth” through maps as @cageycat said but I would love to keep mine whole as long as I can :wink:.
I am not saying we should trash the system or that it is the worst ever, on the contrary it is a great system.
I just think it can be slightly better.

Regardless, of the actual solution TN would use (when resources availability allow it) to help us optimize time on the site, I would be very appreciative for TN for doing so. and I think it would result in faster campaigns.

A big thank you. (especially if you read the whole thing through :wink:)


I have some UI suggestions that might be simple to implement (again I dare not assume anything about the way the system is built or other constraints.)

  1. instead of black only, color the unvisited tiles according to the current CR priorety matrix or a simplified form of it.
  2. “no data voting” button and ability to toggle visibility of empty tiles on and off. a ranking system ( CR maybe?) might be used to filter out ‘spammers’ and the like.

#19

I realize we have new members joining the forum. I want folks to understand the constraints and why this topic goes nowhere.

DG invested a lot of time and money to make the platform. They began hearing suggestions within the next months. They said they’d keep the suggestions in mind. But there would be no changes made to the platform then (or now).

Happy and now Mel, along with 3 Mods (now 2), have explained many times that DigitalGlobe is a small company doing great things. I guess I don’t understand having to explain that Tn’s resources are spread thin (about 1 & a half people). Very simply: The company does not have the ability to reprogram the platform at this time. They ARE listening, but it just is not in the works for now.

Every one of us is a “resource”. All of us have time crunches. Each of us can choose the campaigns we do or choose not to do. We all hope to improve the world from the comfort of our homes. Some of us have more time than others have; many Nodders are older but with illnesses or disabilities. Personally, I used to have more time to nod than I do lately. Tn would love if I had more time… but hey, we all have lives and “work”.

Tomnod knows user behaviors, which went into making the platform we have (and will inform the next platform in the future). By behaviors, in case others are wondering, it means Tn knows that I go to great lengths (and waste time LOL) just to find and start at a corner piece. They know whether I prefer to go R or L on rows, or up and down. The Tn computer can tell how long I plod across a row, tile by tile, along with how much and where I tag. These “behavior” items help the office make the experience as good as they can for us.

But even though they want our suggestions, it is beating a dead topic to keep insisting they change the platform. Not gonna happen right now. It’s on their Big Wish List! They Dream Big! They have Wants, too!

But not right now…


#20

@cageycat. Thank you. I fully understand and acknowledge.:smile:

I knew you would come to scratch my nose about it :slight_smile: But could not resist to propose a solution to a tiny bittsy issue I just figured out ( Did anyone say ADHD?! :wink:).

:slight_smile: P.S. I accept all the scratches with much love :heart:
Hope to be able to get a pat on the back for my late night rumblings one day (or even just a :cat: purr) :wink: