Deep Web Mysteries and Resources. How to Dig undiscovered Gold?

[Saran999 288]

Why this topic? Mainly for culture, pushing knowledge through the modal boundaries where most of us resides.

What is the Deep Web and why I want to talk about it?

 

Relying only on Google or some other search engines is not useful anymore. Searching the web for education, work, leisure, relying only on some well known search engine means to be completely overwhelmed by the 'sois disant' SEO experts, with their blatant and completely unuseful crap.

Some of you are aware of this and dig down Usenet to find some software or information. But the Deep Web is something that is there, not touched by the search engines, and full of info. Full of Gold. Gold that may be just valuable as is, simply publishing it on a personal blog about a definite topic, asking for donations. The only blog with real 'semi' unique information, toward the many millions of others that simply copy their content over and over, even without intelligence, and ask for money - that rightly will never come - and complaining about that. A that's for starter. But there are really tons of ideas on how to employ unknown/not well known information on the web, and that's the side topic for this discussion.

 

Just talking about Usenet between us, IT's old guns that are here since the beginning and where used to connect with analog modems and old stuff like that at 300 Bauds, dreaming of the 'pirate' that had the 1200 Bauds modem and was downloading games for the BBS, may be useful for the youngsters that know only about Bing and Google and Yahooo, don't remember Lycos, or are unaware about Metacrawler. So, this is something that, even if not really "deep web", may be of some usefulness. Talking about Usenet, other search engines and even gopher and veronica, why not?

 

But moving farther, I'll add that, as we are facing an economic crisis worst than the Great Depression and the Big Deal, trying to find useful and unknown information resources may be something worthy to differentiate ourselves and gain incomes. But, and this is the most important point, being able to search for this information is even more valuable. As an old saying recite:'it's better to learn how to fish than to find a single fish'. And I think that we all know how valuable may be another skill under our bellies.

 

So, this is a sort of companion topic to the one on the bot creation and programming, as the searching the Deep Web skill will greatly be enhanced if we own some good bot programming idea that may speed up the process. Better, if this idea is something new and unique. I've dreamed about Matrix and Neo's programming skills that have permitted him to simply sleep in front of his monitor during a custom spider search on Morpheus. Am I alone on that? Why not to learn THAT skill?

 

Obviously, nobody will post the 'one billion dollars' idea 'per se', but I think that discussions, a sort of public/semi public brainstorm on a subject, may improve our possibility to come out with something useful, worthy of our time.

 

So, let's start eliminating all the 'onion' sites and the Dark Net as I do not want to push the discussion on illegality and outlaw content. I like this site as 'a family site', and this is a real insurance that I subscribe, and worthy to defend. So, this topic is absolutely banned since the beginning.

 

I wish only to push an idea based on the percentage of web pages that a great, the most great for someone, search engine like Google has in its database, toward the many times more web content it's present, and waiting for us, out there. I will not search for precise data but the difference is enormous. Tons of resources out there that simply are not visible, or are not, yet, indexed by the 'official' web spiders/crawlers.

 

I'm not an US citizen, but I've been influenced by Your culture and behavior, that I love and respect. And I think that a similar situation is the one very near the first, and the many other, Gold Rush or Oil Rush, or Whatever Rush that have characterized Your economic history and that started just after a huge economic crisis, with the poor, the dead, the hunger and so on. Many enterprises are still beating on primary resources, but i think that the most smart way to proceed for us, that don't have trillions of dollars to spend on R&S, may be to push for hidden, not well known, unknown information. Scarcity makes value, so... the bottom line is very simple.

 

Wanna talk about it? (where is the 'green' smile?)

[gustav2007 18]

So how and where do we find this undiscovered Country??

 

[zin48 1]

  I was having flashbacks of the early 90's when you were talking about old technology.  I remember having a cd rom drive for the first time and upgrading the modem in that computer to 14400baud.  I thought I was flying around the net.  Most of us that were on the net back then used Lycos and webcrawler.  It was pre Google and the search engines seemed to find whatever we looked for at the time.  I still have the 9600 baud modem I took out of that pc and my IBM ps1 in my garage.  It seems that much of the knowledge on how to find hidden information on the net has been lost or not shared with as many people as back then.  I think it comes back to people thinking knowledge is power.  The truth is that knowledge is power but only if you have people to share it with and improve on the ability to acquire that knowledge and use it for something productive and not destructive.  I used to write bots for games back in the day like Diablo and Diablo II but have not done anything like that in years as it became a burden and was abused by so many.  In moderation it is not bad but when it is a huge part of a gaming community it destroys games.  This ruins any fun for those that do not wish to participate in the botting community.  I got tired of all these teens saying I am a hacker and tired of watching people do malicious things online so I stopped participating in something I did for fun and not personal gain.  The "deep web" as some call it is full of people that are there because they are tired of people exploiting what they do for their own personal gains.  There are also people on the other side of the coin but they also do not exploit and mass distribute in the open.

  There is so much more information there but you have to be careful where you go and what you do.

 

 

edited for bad typo