« September 2004 | Main | November 2004 »
October 31, 2004
P2P Traffic Measurement
After our decentralized filesharing post we discovered this item from ACM News Service, "Is P2P Dying or Just Hiding?"
High-order bits excerpted from the paper itself:
- In our traces, P2P traffic volume has not dropped since 2003. Our datasets are inconsistent with claims of significant P2P traffic decline.
- We present a methodology for identifying P2P traffic originating from several different P2P protocols. Our heuristics exploit common conventions of P2P protocols, such as the packet format.
- We illustrate that over the last few years, P2P applications evolved to use arbitrary ports for communication.
- We claim that accurate measurements are bound to remain difficult since P2P users promptly switch to new more sophisticated protocols, e.g., BitTorrent.
More bits:
CAIDA monitors capture 44 bytes 2 of each
packet (see section III), which leaves 4 bytes of TCP packets to
be examined (TCP headers are typically 40 bytes for packets that
have no options). While our payload heuristics would be capable
of effectively identifying all P2P packets if the whole payload
was available, this 4-byte payload restriction limits the number
of heuristics that can undoubtedly pinpoint P2P flows. For example,
BitTorrent string “GET /torrents/” requires 15 bytes of
payload for complete matching. Our 4-byte view of “GET ”
could potentially indicate a non-P2P web HTTP request.
The ACM News Service summary...
"Is P2P Dying or Just Hiding?"The full article (pdf) is contains many more bits for the interested reader...
CAIDA.org (10/04); Karagiannis, Thomas; Broido, Andre; Brownlee, NevilUC Riverside's Thomas Karagiannis, the Cooperative Association for Internet Data Analysis' (CAIDA) Andrew Broido, et al. dispute popular media reports that peer-to-peer (P2P) file-sharing has declined precipitously in the last year, and contend that the reverse is actually the case. The authors attempted to measure P2P traffic at the link level more accurately by gauging traffic of all known popular P2P protocols, reverse engineering the protocols, and labeling distinctive payload strings. The results support the conclusion that 2004 P2P traffic is at least comparable to 2003 levels, while rigid adherence to conventional P2P traffic measurement techniques leads to miscalculations. The percentage of P2P traffic was found to have increased by about 5 percent relative to traffic volume. Furthermore, comparisons between older and current P2P clients revealed that the use of arbitrary port numbers was elective in older clients, while current clients randomize the port number upon installment without the need for user action. Meanwhile, P2P population studies found that the ranks of IPs grew by about 60,000 in the last year, and the number of ASes participating in P2P flows expanded by roughly 70 percent. These findings outline several trends, including evolving tension between P2P users and the entertainment sector; increasing demand for home broadband links; plans to directly induce P2P applications into profitable traffic configurations; and a significant transformation in supply and demand in edge and access networks, provided that P2P traffic maintains its growth and legal entanglements are eliminated.
Posted by rohit at 12:10 PM | Comments (0) | TrackBack
October 30, 2004
Amazon Lets Customers Paste Photos
CNET refers to an online trend -- "the blurring of e-commerce and personal media such as Web logs and social networking sites":
Amazon.com has quietly introduced a new feature on its Web store that lets customers post photos alongside product reviews--its latest effort to build a sense of community among customers.The e-tailer introduced the feature, called Customer Images, last month for certain product categories including electronics, apparel, sporting goods and musical instruments. It added kitchen items, tools and hardware on Tuesday. The feature is in beta, meaning the company is still testing and tuning it.
"This feature allows customers to really showcase how they are using the product," Amazon spokesman Craig Berman said. "It's a great addition to our customer experience."
The idea is to let customers highlight specific attributes of a product, such as size, and show the product in action, he said.
The Web and Web-based Services are finally discovering how to personalize their content to their customers, and it's thrilling to watch this trend unfold.
Posted by adam at 02:31 AM | Comments (1) | TrackBack
October 29, 2004
Smart Health
Today CommerceNet's Smart Health Portal was launched during the Smart Health Summit. The initiative:
Smart Valley, a non-profit committed to turning Silicon Valley into the leading user of the technologies it invents, and CommerceNet will work with the Smart Health Forum, a community of healthcare provider, insurers, employers and foundations in Silicon Valley. The Forum, which is open to all interested parties, will collaborate on the implementation of network-based tools that will greatly facilitate the sharing of medical records without compromising patient privacy, data security or provider choice.The network will allow patients and providers to ‘publish’ an entry in a data registry when a healthcare transaction occurs, whether it’s a visit to a provider, filling a prescription or getting a test result. All of the patient’s data remains in the source systems – only a pointer is recorded. The next time there is a visit, whether to a doctor’s office or an emergency room, authorized caregivers will be able to see the transactions and call up x-ray images, test results and other information needed to provide rapid diagnosis and treatment. Similarly, consumers will be able to see and update their personal information and that of their children. Patients and providers will set the policy that determines who can see what data.
These network tools will be piloted in several different environments. Projects being considered include a senior- and chronic care in the emergency room and a ‘Virtual Clinic’ concept that illustrates how physicians and community-based caregivers can share information.
The pilots will be funded through a collection of public and private funds. CommerceNet is applying for state and federal funds, some of which will require matching funds from local organizations. We will work with community-based foundations to help us raise funds.
When proven, Smart Health will be made available to the rest of the Bay Area, California, and the nation.
Posted by adam at 12:43 PM | Comments (0) | TrackBack
October 28, 2004
WikiHealth
Brian Dear has an intriguing idea:
Imagine WikiHealth or WikiMed, an open, collaborative health database written by everyone in the world? If millions of people could contribute articles on health and well-being, diseases, treatments, symptoms, remedies, and personal experience with what worked and what didn't with prescriptions, would the world be better off? Are you currently happy with the state of medical knowledge on the web? If you or someone you know is suffering from some condition, and you type the name of the condition into Google, are you satisfied you're getting good results? In an age where, at least in the U.S., doctors are less and less likely to give you the time of day let alone spend time with you going into detail about everything there is to know about a condition, wouldn't it be useful if there were an online resource with a strict NPOV (neutral point of view) containing in-depth encyclopedia information about health-related subjects?WikiHealth. WikiMed. (Don't bother, the domains are taken -- maybe there's hope!) But you get the idea: a worldwide open collaborative compendium of practical health and medical knowledge. Isn't it time such a service existed?
Yes. Yes it is. For another example, there's a related idea floating around CommerceNet's healthcare team that it would be interesting to drop the NPOV and offer parametrized search -- imagine if, for more controversial complementary-medicine theories, patients/advocates could just submit their anecdotal stories, but then you could ask for stories "similar" to your own to see what's out there. In any case, WikiMed could be quite personalized to your medical "ideology" and idiopathy...
[Kind of like how the CreativeCommons search engine is only a slight, but suggestive, parametric constraint on ordinary Nutch searches]
Posted by adam at 08:51 AM | Comments (0) | TrackBack
RFID Phones
RFID cell phones take shape at Nokia
For instance, retailers could put RFID-embedded "touch phone here" signs on store shelves to send a coupon to the phone, or put the same signs at checkout stands to instantly transfer personal information stored on the phone in order to complete a warranty, Nokia Director Gerhard Romen said.At the CTIA Wireless IT & Entertainment trade show here, Nokia was demonstrating an early prototype built in collaboration with VeriSign, which is proposing a central repository for RFID data that companies can use to relay information about inventory and deliveries to customers and suppliers. The prototype was based on Nokia's 5140 model, with an RFID reader contained in a shell attached to the phone.
"It's still very early yet," Romen said Sunday when asked when RFID phones may become commercially available.
Posted by rohit at 08:47 AM | Comments (1) | TrackBack
October 27, 2004
Plott's Cool Experiment
Science News: Best guess: economists explore betting markets as prediction tools
The research that led to future-predicting markets stems from the 1960s and 1970s, when Vernon Smith and Charles Plott, now of George Mason University and the California Institute of Technology in Pasadena, respectively, began using laboratory experiments to study different market designs. In the early 1980s, Plott and Shyam Sunder, now of Yale University, tested how well markets aggregate information by designing a set of virtual markets in which they carefully controlled what information each trader had.In one experiment, Plott and Sunder permitted about a dozen study participants to trade a security, telling them only that it was worth one of three possible amounts--say, $1, $3, or $8--depending on which number was picked by chance. Plott and Sunder then gave two of the participants inside information by telling them which amount had been selected. Traders couldn't communicate with each other; they could only buy and sell on the market.
"The question was, Would the market as a whole learn what the informed people knew?" Plott says. "It turned out that it would happen lightning fast and very accurately. Everyone would watch the movements of the market price, and within seconds, everyone was acting as if they were insiders."
In another experiment, Plott and Sunder gave the inside traders less-complete information. For instance, if the outcome of the random pick were $3, they would tell some traders that it was not $1, and others that it was not $8. In these cases, the market sometimes failed to figure out the true value of the security.
However, if Plott and Sunder created separate securities for each of the three possible outcomes of the random pick instead of using one security worth three possible amounts, the market in which some traders had incomplete tips succeeded in aggregating the information.
The studies established that, at least in these simple cases, markets indeed can pull together strands of information and that different setups affect how well they do so.
This type of experiment gave researchers a "wind tunnel" in which to test different market designs, says John Ledyard, a Cal-tech economist who chairs the board of Net Exchange. "With experiments, we're starting to zero in on what really works," he says.
Posted by rohit at 08:39 AM | Comments (0) | TrackBack
October 26, 2004
Negroponte on p2p ecash
Some people, including myself, believe the next step is for some of those bits to have value. That is to say, consider a string of bits to be like a virtual cow or shell. In order to distinguish these bits (like telling the difference between a beautiful seashell and a piece of coal), they would need an agreed identity. To avoid forgery, they would need a unique and secure ID. And to stop multiple spending of the same bits, there would need to be a clearing process or a means to reveal the identity of anybody who tries to double-spend. All of these requirements are easily achieved in both traceable and anonymous systems of E-cash. In these cases, the money does move. The bits are money. The more you have, the richer you are. This is the future, though maybe only in part.A parallel and more intriguing form of trade in the future will be barter. Swapping is a very attractive form of exchange because each party uses a devalued currency, in some cases one that would otherwise be wasted. Many of us are too embarrassed to run yard sales or too lazy to suffer the inconvenience and indignity of eBay. But imagine if you weren't. The unused things in your basement can be converted into something you need or want. Likewise, the person with whom you're swapping is giving something of value to you which is less so to them. With minimal computation, three-way, four-way, and n-way swaps can emerge, thereby removing the need for any common currency.
Swapping is extended easily to baby-sitting for a ride to New York, a mansion for a two-hundred-foot yacht, or leftover food for a good laugh. In some cases, people will swap for monetary or nonmonetary currencies. Without question, we'll see new forms of market-making and auctions. But the most stunning change will be peer-to-peer, and peer-to-peer-to-peer- ... transaction of goods and services. If you fish and want your teeth cleaned, you need to find a dentist who needs fish, which is so unlikely that money works much better. But if a chauffeur wanted fish and the dentist wanted a driver, the loop is closed. While this is nearly impossible to do in the physical world, it's trivial in cyberspace. Add the fact that some goods and services themselves can be in digital form, and it gets easier and more likely. An interesting side benefit will be the value of one's reputation for delivering on your promises -- thus, identities will have real value and not be something to hide.
The point can be generalized beyond money. Peer-to-peer is a much deeper concept than we understand today. We're limited by assumptions rooted in and derived from the physical world. Information technology over the next 25 years will change those limits through force of new habits. Let me cite just one: I think nothing of moving millions of bits from one laptop to another (inches away) by using the Internet and transferring those bits through a server 10,000 miles away. Imagine telling that to somebody just 25 years ago.
Nicholas Negroponte is the founding chairman of MIT's Media Laboratory and the author of the seminal work on the digital revolution, "Being Digital" (Knopf, 1995).
Posted by rohit at 03:35 PM | Comments (0) | TrackBack
October 25, 2004
Kev's Color Picker and SVG Notes
Kevin Hughes wrote a beautiful Color Picker application in about 100k of SVG. The code is Creative Commons-licensed.
More telling are Kev's notes on SVG, in which he tells the good, the bad, and the ugly. His recommendations:
Looking ahead, there could stand to be efforts for making today's dynamic languages ready for real GUI application development using SVG. This mostly means adding support for multithreading, Unicode, and internationalization. Once these things are well baked into PHP or other dynamic language, there is no stopping the new application development paradigm. With PHP support you'd convert hordes of developers immediately. Applications could run on the Web and be deployed on the desktop with the same code, and could be created and edited by just about anybody, if you wanted them to. Every application could print in full PDF quality. Sounds a bit like the promise of HyperCard, but with vector graphics...With SVG we are now at the edge of a new shift in how people think about applications, just as we were when the Web first started. We've had a long enough fight trying to get our content out of proprietary data jail formats, and sure, we have a ways to go - but every HTML page I wrote in 1993 still renders the same in modern browsers on modern computers on every major platform; with how many programs can you say that? Let's try to do the same for applications and help make them more future-proof with SVG.
Posted by adam at 11:32 AM | Comments (0) | TrackBack
October 24, 2004
Web Services Commons
Nice analysis by Joyce Park on Web Services Commons:
Flickr is a company that has gotten into open web services in exactly the right way. It costs them a lot of money to host all that bandwidth, but feeds of photos are a cool new feature and a value-add to their users -- and ultimately it all loops back to their core business model of getting people to host photos on Flickr. Flickr's API also gives users some confidence that the company isn't going to try to screw them by making it hard to get their content back out if they want to take it and go elsewhere. It's a win-win-win situation for everyone, and another example of Flickr's leadership by clarity.On the other side of the spectrum are companies that treat RSS feeds and public web services just as free content, without adding any new transformative value or giving anything back to the community. There's nothing necessarily wrong with this if the license allows it, but it's not interestingly different from screenscrapers who present your content as theirs -- who have long been considered sleazy parasites by most of the legitimate web. The whole idea of opening content via web services is that growth for all can be enhanced by sharing -- and in the long run people don't want to share with those who are openly contemptuous of the whole idea.
Opening content via web services used to be the Holy Grail, but slowly the world is rewarding the owners of services that do just that, as the Flickr example illustrates.
Posted by adam at 11:23 AM | Comments (0) | TrackBack
October 23, 2004
ACM Conference on Electronic Commerce
ACM Conference on Electronic Commerce (EC'05)
ACM Conference on Electronic Commerce (EC'05)
June 5-8, 2005
Vancouver, Canada
Vancouver Marriott PinnacleSociety sponsor: ACM Special Interest Group on E-Commerce (SIGECOM)
Conference Web Site: http://www.acm.org/ec05/Since 1999 the ACM Special Interest Group on Electronic Commerce (SIGECOM) has sponsored the leading scientific conference on advances in theory, systems, and applications for electronic commerce. The Sixth ACM Conference on Electronic Commerce (EC'05) will feature paper presentations, workshops, and tutorials covering all areas of electronic commerce. The natural focus of the conference is on computer science issues, but the conference is interdisciplinary in nature, addressing the following topics:
Algorithmic mechanism design
Auction and negotiation technology
Automated shopping, trading, and contract management
Computational finance
Computational game theory and economics
Computational markets for information services
Databases and online transaction processing
Experience with fielded electronic-commerce systems
Formation of supply chains, coalitions, and virtual enterprises
Information markets
Intellectual property and digital rights management
Languages for describing goods, services, and contracts
Legal, political, and social issues
Marketing and advertising technology
Payment and exchange protocols
Recommendation, reputation, and trust systems
Security and privacy issues in electronic commerce
Software and systems requirements, architectures, and performance
User-interface issues in electronic commerceThis list is meant to be representative but not exhaustive.
The conference will be held from Sunday June 5th through Wednesday June 8th 2005 in beautiful Vancouver, Canada at the Vancouver Marriott Pinnacle.
Tutorials and workshops will be held on Sunday, June 5th. Papers and invited talks will be presented beginning on the morning of Monday, June 6th and running through noon on Wednesday, June 8th. More detailed program and schedule information will be released as it develops.
Posted by rohit at 02:54 PM | Comments (0) | TrackBack
Web Services Constraints and Capabilities
"W3C Workshop on Constraints and Capabilities to Explore Next Web Services Layer"
XMLMania.com (10/12/04)World Wide Web Consortium (W3C) members are working on a Web services constraints and capabilities framework that will allow organizations to communicate the terms of their service. Requirements for using HTTP or the ability to support GZIP compression, for example, need to be communicated in a standard manner using the Web Services Description Language (WSDL 2.0) specification, SOAP, or HTTP. W3C director Tim Berners-Lee said more standards were needed to support automated Web services. Participants at a two-day W3C workshop on Web services constraints and capabilities were required to write a position paper stating how they would preferably communicate constraints and capabilities in regards to reliable messaging protocol requirements, encryption using WS-Security or other security mechanisms, and an attached P3P privacy policy. Besides discussing how best to implement such constraints and capabilities requests and what vocabularies to use, the workshop participants also discussed their framework's impact on and relation to other W3C protocols and Web technologies. Upcoming W3C recommendations include WSDL 2.0, which is currently in the "last call" stage and will be extended by the constraints and capabilities framework. SOAP 1.2 is also nearing completion, having advanced to candidate recommendation status in August 2004.
Posted by rohit at 02:53 PM | Comments (0) | TrackBack
October 22, 2004
Mmmmm... Verichip...
Mike Langberg of the SJ Mercury News:
I'm rolling up my sleeve, ready to get injected with the VeriChip. That's the device cleared by the U.S. Food and Drug Administration earlier this month as the first implantable electronic identification tag linked to a person's medical profile...VeriChip is not new technology. Applied Digital of Delray Beach, Fla., which developed the product, has sold 30 million virtually identical injectable ID chips for pets and livestock in the past 15 years. Thousands of lost dogs and cats without external ID tags have been reunited with their owners because the pets had ID chips that were scanned at animal shelters.
VeriChip is very simple. A glass capsule about the size of a grain of rice, the VeriChip contains only a basic microchip programmed with a 16-digit ID number and an antenna. When the VeriChip is hit with radio waves from a scanner, the chip responds by broadcasting its ID number. There are no batteries in the VeriChip, and it works for a lifetime...
We need basic protections, similar to what's already in place for other technologies ranging from home phone lines to financial records. I'd humbly suggest federal laws covering five points:
1. Getting an implanted ID chip should always be voluntary, without coercion. No one should ever, ever be forced to get an ID chip. What's more, schools, businesses and government should be banned from offers that pressure people -- prisoners, for example, couldn't be promised time off their sentences, and insurance carriers couldn't offer lower rates.
2. There has to be an ``off'' switch. Implanted chips must either have the capability to be permanently turned off, or be removed. The VeriChip can't be deactivated, but can be removed in a minor outpatient procedure. Deactivation or removal should be available on demand, no questions asked, and should be free; the fee for inserting a chip should include a reserve fund to pay for removals.
3. Scanners can't be hidden. The scanners that read implanted ID chips can be built into walls, door frames or even highway signs. There needs to be a universal symbol showing the location of ID scanners, and that symbol must be shown wherever a scanner is present.
4. Individuals must be in control. Chip recipients need full disclosure and absolute veto power over what information goes into computer databases tied to their ID number, and who has access to that information. Again, schools, business and government couldn't use coercion -- such as an employer who sets up scanners within a company building, then insists as a condition of employment that workers allow scanning of implanted chips to track their movement.
5. Government can't snoop with a court order. Law enforcement agencies would need to convince a judge of their legitimate interest in looking at your ID database in the same way they need court orders today to look at phone or bank records.
Posted by adam at 02:19 PM | Comments (0) | TrackBack
October 21, 2004
Decentralization Defined
We've spent a little time working on the Decentralization page in the CommerceNet Labs wiki. Here's a snapshot of what we have so far:
Decentralization in Commerce means the freedom to do commerce the way you want, rather than the way your software wants.We believe that to build software that works the way society works, that software design must reflect the principles of decentralization.
An agency is an organization with a single trust boundary. One way to think about decentralization is that it allows multiple
agencies to have different values for a variable.See also:
Update. Allan Schiffman notes that Webster's has a fine definition of decentralization as well: "the dispersion or distribution of functions and powers". I'll add that to the wiki as well...
Posted by adam at 03:55 PM | Comments (2) | TrackBack
October 19, 2004
zSearch and CommerceNet's Neighborhood
I want SupplyFX, Webify, and Bonsai Development to be in CommerceNet's Neighborhood. I also want Rob Rodin, Allan Schiffman, Kevin Hughes, and Marty Tenenbaum in CommerceNet's Neighborhood.
And by linking to those pages like I just did, I just did. Such is zSearch.
Posted by adam at 01:21 PM | Comments (0) | TrackBack
October 18, 2004
"Fluffy Bunny" is a WinSock-puppet
We love Google's new Desktop Search. We've been arguing about something like this for a year or more. The idea of searching everything you've seen — not just your hard drive, but everything hyperlinked to it (such as your surfing history) — is so intriguing we've built something similar for ourselves.
We've modified Nutch to search not just CommerceNet's website, weblog, and wiki, but also everything we link to. Go ahead and try our index of CommerceNet's Neighborhood. If you query for Nutch you won't just see pages from our sites, but Nutch's home page and even an application of it at CreativeCommons...
We're also exploring a bunch of other ideas that GDS and other desktop indexing projects like StuffIveSeen haven't tackled yet. Foremost is ranking -- just like the AltaVista engine that Google itself dethroned, GDS doesn't have anything like a PageRank for the gigabytes of information on your disk. Like many researchers, we suspect that a users's social network is the key to discerning which hits are likely to be most useful. After all, if the Web is drowning in infoglut on any given query term, a user should be such an expert on the terms of his or her art that there ought to be even more hits to rank on localhost. One cure may be collaborative filtering with your friends...
Secondary aspects of the problem include tackling the fact that many of us have multiple computers and identities on the Internet, so we'd need networks of personal search engines. Or that a local-proxy-server approach might be better at capturing the ''dynamics'' of our interaction (how often we re-read the same email over IMAP, say).
But rather than rattling off a longer list of half-baked hypotheses, I'd like to cite GDS for at least one idea that never occurred to us: integrating it seamlessly with the public site. Sure, we thought AdWords-like ads were the key to a better revenue model for the Fisher category of PersonalWeb products.
No, what's cool is that Google's ordinary results pages from the public website automatically include hits from your hard drive. How'd they ''do'' that?! Read on...
CommerceNet Labs Wiki : FluffyBunnyBurrowsIntoWinSock
...we found that Google Desktop Server actually hooks into Windows' TCP/IP stack to directly modify incoming traffic from Google's websites to splice its local results in. Once you install GDS, there's a bit of Google's code running inside every Windows application that talks to the Internet.It's done using a long-established hook in WinSock2, its Layered Transport Service Provider Interface (SPI)...
The Winsock LSP is mostly only used by spyware and censorware; it's a surprise to see a positive use for it. Spyware detectors like HijackThis consequently detect it.
[An aside: why is Rifkin's GLAT posting more relevant on the query "rifkin fisher" than Rifkin's actual Fisher posting? I think it's Battelle's fault, for increasing the GLAT's PageRank! :-]
Posted by rohit at 09:03 PM | Comments (0) | TrackBack
October 17, 2004
There Is No Cat
Using Desktop Google I found an email from Ross Stapleton-Gray (author of An Internet of Less Than Solid Things, among others), saying
zCommerce is to eBay, as radio is to telegraph... ("eBay, except there's no cat!")
citing the Einstein cat story:
You see, wire telegraph is a kind of a very, very long cat. You pull his tail in New York and his head is meowing in Los Angeles. Do you understand this? And radio operates exactly the same way: you send signals here, they receive them there. The only difference is that there is no cat.--Albert Einstein, when asked to describe radio.
Posted by adam at 02:45 PM | Comments (0) | TrackBack
October 16, 2004
Network-Attached Processing?!
One immediately wonders if NAP is well-designed for RESTful applications... :-)
Looks like Bob Pasker has resurfaced happily after Kenamea:
Although J2EE has become the defacto standard for business application development, companies still face issues with reliability, predictability, and compute power...Azul has created a groundbreaking new class of compute infrastructure. Network attached processing seamlessly delivers on the dream of mainframe-class capabilities at new world economics." -- Bob Pasker
Co-Founder; Chief Architect – WebLogic; Java Luminary
Azul Systems: Products & Solutions
The key to network attached processing is Azul virtual machine proxy technology. This patent-pending technology, initially targeted at Java and J2EE platform-based applications, transparently redirects application workload to the compute pool. No changes are required to applications, or the existing infrastructure configuration. The Azul technology works with J2EE platform products including BEA WebLogic and IBM WebSphere application servers. Compute pool appliances are simply connected to the network and Azul software is installed on the application hosts. Suddenly every application has access to a virtually unlimited set of compute resources.
Each compute pool consists of two or more redundant compute appliances—devices designed solely to run massive amounts of virtual machine-based workloads. Each appliance has up to 384 coherent processor cores and 256 gigabytes of memory packed in a purpose-built design that delivers the benefits of symmetric multiprocessing with tremendous economic benefits. The massive SMP capacity of these appliances enables applications to dynamically scale, responding to varying workload and spikes without the pain of having to reconfigure or provision application tier servers. The targeted design provides small unit size, high rack density, low environmental costs, and simple administration.
Posted by rohit at 12:22 PM | Comments (0) | TrackBack
October 15, 2004
OpenSheet
Steve Wilhelm served as the development manager for Tibco's MarketSheet for Windows, which was a Reuters-inspired spreadsheet for real-time information. Using those insights and looking at what has changed in the years since, Steve describes a hypothetical product called OpenSheet that takes into account the advances made in mobile computing, social networking, and open source in the last half decade.
As described by Steve, OpenSheet is a compelling product vision, and it makes me wonder: could there be a new kind of spreadsheet for real-time data that takes a three-dimensional view rather than the traditional spreadsheet table view, so that you could see the history of a stream? In other words, a "StreamSheet" visualization?
Posted by adam at 10:42 AM | Comments (1) | TrackBack
RFID Under Skin?
The New York Times > Technology > Identity Badge Worn Under Skin Approved for Use in Health Care:
Real privacy concerns have emerged. "At the point you place the chip beneath the skin, you're saying you will not have the ability to remove the ID tracking device," said Marc Rotenberg, executive director of the Electronic Privacy Information Center, a public interest advocacy group in Washington. "I think, increasingly, if this takes off - and it's still not clear that it will - the real social debate begins around prisoners and parolees, and perhaps even visitors to the U.S. That's where the interest in being able to identify and track people is."Indeed, the debate over civil liberties and privacy has made discussing any practical benefits of a technology like VeriChip harder.
"The fact that we're engaged in such a deep, fundamental privacy debate really does complicate the prospect for this kind of technology," said Clyde Wayne Crews Jr., director of technology studies at the Competitive Enterprise Institute, a regulatory research group in Washington. "We haven't even sorted out the appropriateness of a RFID tag that goes on a pallet of tomatoes," Mr. Crews said, "much less one that can go under a person's skin."
Posted by rohit at 10:11 AM | Comments (1) | TrackBack
October 14, 2004
The difference between Web and Middleware
Mark Nottingham wrote:
From the standpoint of interface semantics, the difference here is really just one between saying “POST machineMgmtFormat” and “MANAGEMACHINE.” In the uniform approach, the service-specific semantics are pushed down into the type (media type) and content (entity body) of the data. In the specific approach, they’re surfaced in the interface itself.This isn’t a big difference... While there isn’t much technical difference between these approaches, there’s a big gap in how people can use them. In HTTP, creating new methods is expensive, while creating new URIs is very, very cheap. OTOH, Web services makes creating new methods cheap, while making the creation of new URIs expensive. This is not because either approach is technically limited; it's due to the design of both the specifications — like WSDL — and the toolkits that people use.
Bill de hÓra adds:
While HTTP does allow for addition, practically speaking, the verb set is fixed. It has taken years for WebDAV additions to HTTP* to penetrate more than a fraction of the Web. Other efforts, such as HTTPR, an extension for reliable messaging, have gone nowhere. Even within the mandated verb set of HTTP itself, we find the availibility of verbs varies widely (notably PUT and DELETE) with entire eco-systems (such as mobile device clients) having only a subset. One can argue that the active verb set of HTTP comprises a subset of 3 verbs - HEAD, GET, POST - anything else is dead tongue.The problem with HTTP POST, and what makes it special, is that it is a semantic catchall. What makes POST a uniform speech act is ironically the absence of interesting semantics and lack of specificity. Although it has specifications that are helpful to people when dealing with caches and state management, there's no controlled means of defining what one is actually saying with it, without some further and prior agreement between client and server. The reality is that POST has been overloaded and abused to get systems talking even where such systems would have done better with an alternate verb - and the result is that in many systems the POST speech act is close to meaningless. WS-Transfer aims to throw some light into this void by providing a means to add consistent meaning to operations that would often be drilled through POST. In particular this may prove valuable for use with web services toolkits which are often designed to hide the networking aspect of communications from the developer.
...
Systems are being built, week in, week out, than cross the Web/Middleware boundary without being informed by both approaches and where they are approriate. This implies projects with excess risks and costs, wasted effort, re-learning of best practices or what is already in the state of the art. This is all the more important now that systems that incorporate web and middleware aspects are increasingly the norm (the size of the industry sector affected is significant).
At CommerceNet Labs we have a team of people who jump back and forth between the Web and middleware worlds regularly, and none of us find ourselves enthusiastic about WS-Transfer. I'm trying to figure out why that is. Perhaps it's Mark Nottingham's Web services has no architecture observation.
Perhaps it's the potential for verb proliferation getting out of hand. Perhaps it's just general WS-* malaise. Or perhaps it's the feeling that this protocol feels more complicated than HTTP for most applications that come to mind. Services shouldn't have to be complicated, right?
Posted by adam at 11:50 AM | Comments (0) | TrackBack
October 13, 2004
How many products are there?
Sergei Burkov of Dulance asked via email:
Any idea how many products are out there?This can probably be derived from the total number of UPC codes issued, or something. (We can probably ignore products for which no code of any type has been issued.)
Is there an estimate how many different products are sold on the world's markets today?
Ross Stapleton-Gray responded:
According to Peter Hurtubise at QRS, the QRS Catalogue is now up to 100 million entries (up from c. 80 million a few years ago). NB that QRS is GMA-focused [GMA being General Merchandise and Apparel], so some large swath of that will be every variation of every size of a particular article of clothing, say.I've a couple of my own questions that would help illuminate this space; most could probably be answered by a QRS, or one of the major retailers, but they'd unfortunately see them as rather sensitive. A big one would be (for Target, or Wal-Mart): what is the total SKU or product code count for what you carry (my guess is that it's in the several tens of thousands... perhaps several times higher if they've got a lot of apparal)? Of that, how many manufacturers would account for 90% of the total product count? Of 99%? My guess is that a relatively smallish number (say, 500) might produce the 90% coverage. That's distinct from (though these would be other numbers worth knowing) the *unit* count of goods inventoried/sold.
And so we continue the search for an answer of how many products are out there...
Kragen notes that most products don't come from companies or even people rich enough to be assigned product codes, as related to his essay on the long tail of ecommerce. Also, if we take into account that, for example, in a pack of colored markers each marker should have its own product code (and not just the pack), or that a Spiderman-contest-branded can of Diet Dr. Pepper should have a different product code than a regular can of Diet Dr. Pepper... well, there are likely to be at least an order of magnitude more products than we think of as "product coded" today.
Posted by adam at 10:50 AM | Comments (3) | TrackBack
October 12, 2004
ScoutPal
Cool use of Amazon Web Services: ScoutPal...
Use ScoutPal with any web-enabled cell phone or wireless PDA and find out the Amazon Marketplace value of books, CD’s, DVD’s, Video Tapes, or any other Amazon Marketplace Merchandise, while you are out bookscouting.How many times have you come back from bookscouting with essentially worthless books? How many times have you passed over a book, only to find out later, when you look it up again on Amazon, that it was valuable and in high demand?
Bookscouting with ScoutPal is like hunting with RadarYou can quickly comb through stacks of books, zero in on the gems, and find the treasure!
ScoutPal is simple and easy to use. Just enter the ISBNs or UPCs, and ScoutPal will "Fetch" the information you need, and quickly present it to you in a concise form. Results include a summary of market prices and quantities, sales rank, editions and availability, and used/new/collectible details. You can customize the content and format of your results, and switch formats at anytime. You can also be alerted if there are Buyers Waiting.
Jeff Bezos mentioned this company at Web 2.0 as an example of the kind of one-person innovative offering that Amazon Web Services unleash.
Posted by adam at 10:01 AM | Comments (0) | TrackBack
October 11, 2004
Forex market nearly $2T/day
It's official -- or as official as it's going to get in such a decentralized market -- the daily turnover in currency trades is nearly double the oft-quoted trillion-dollar-a-day mark...
The Wall Street Journal version of this story on 9/29 noted that "the surge in trading levels indicates that foreign exchange is becoming increasingly well established as an asset class in its own right as an alternative to stocks and bonds."
Triennial Central Bank Survey of Foreign Exchange and Derivatives Market Activity 2004
"Average daily global turnover in traditional foreign exchange markets rose to $1.9 trillion in April 2004, up by 57% at current exchange rates and by 36% at constant exchange rates as compared to April 2001"
Posted by rohit at 10:50 AM | Comments (0) | TrackBack
October 10, 2004
[NYT] Glushko on Halsey's new fund
It's interesting to see an CommerceNet alum Bob Glushko in the news. The comment, though, is a telling one, since if anything CommerceNet is firmly on the same side as Halsey in its committment to open business service networks. On the other hand, a decentralized aspect of that vision is that you should always be able to run your "own" GrandCentral-like service, too.
Not to mention that we need to all work together to continue to evolve WS interoperability to include real-time, async events -- the need for a two-way GrandCentral with notifications more powerful (& scriptable) than, say, email...
Moreover, there is a nagging question confronting Grand Central and every other company pursuing the on-demand model. Will large corporations ever feel comfortable with a solution that means placing valuable data outside their computer security systems, or will they stick with solutions that remains safely inside their networks and delivered on individual computers?"The Grand Central hosting approach might appeal to small firms that need just a few services, but most big enterprises aren't going to yield this control to a third party," said Robert J. Glushko, an entrepreneur before taking a teaching post at the School of Information Management and Systems at the University of California, Berkeley. "So there's a business model there, but it may not be a very big one."
Posted by rohit at 10:31 AM | Comments (1) | TrackBack
October 09, 2004
Web 2.0
I posted my Web 2.0 Notes on my typepad for anyone who's interested...
Posted by adam at 01:13 PM | Comments (0) | TrackBack
October 08, 2004
Web Service Grids: An Evolutionary Approach
Web Service Grids: an Evolutionary Approach defines "a web service specification profile WS-I+ that builds upon the recognised WS-I Basic & Security profiles with the additional specifications: WS-Addressing, WS-ReliableMessaging and the Business Profile Execution Language (BPEL)."
It might be the right approach for that community, but it's still astoundingly complex.
Posted by adam at 10:48 PM | Comments (0) | TrackBack
October 07, 2004
The long tail of ecommerce
In February, Dave Sifry talked about blog popularity by link count, which his company Technorati tracks:
"Everybody talks about the power law. Fuck it, I've got the data." All the power law says is that when it's easy to publish that you'll have a relatively small number of things (compared to the entire space of options) which are linked to by a lot of people. The important part is not the top 100 (bfd) but what happens in the top 100k when there are five inbound links, which is significant because it means that there is still a community for people.
Here's what's interesting. If this was broadcast and only the big guys mattered, the graph would look the same. But what we're seeing is that the aggregate number of links in the lower portion of the graph greatly outnumber the links into the top 100. There are lots more little clusters than big clusters.
(I haven't been able to find a transcription of the actual talk, so I'm quoting notes.)
In April, David Weinberger wrote an article arguing that the blogosphere isn't an echo chamber for this reason:
David Sifry, the creator of Technorati.com, a site that indexes and ranks 1.6 million weblogs, points out that even though there is a power curve, if you rank blogs by how many sites link to them, the 100,000th blog has five links pointing at it. Five isn't a thousand, but it still means that five people with sites think enough of that 100,000th blog to recommend it to others. Presumably, that site is important to a small cluster of people. That's a readership that didn't exist before the Net. Further, if you add together all of the blogs in the "tail" of the power curve, it's a hell of a lot of blogs and a hell of a lot of readers. So, while the head of the power curve feels familiar to us because it's essentially a bunch of online columnists, the long tail is something new and unfamiliar: a galaxy of people who are finding constellations of readers, ready for ideas and conversation.
It turns out, though, that the internet is enabling long-tail popularity to take off in many different spheres. Wired just published an article entitled "The Long Tail", which talks about aggregate popularity in power-law distributed populations such as songs, books, and movies.
What's really amazing about the Long Tail is the sheer size of it. Combine enough nonhits on the Long Tail and you've got a market bigger than the hits. Take books: The average Barnes & Noble carries 130,000 titles. Yet more than half of Amazon's book sales come from outside its top 130,000 titles. Consider the implication: If the Amazon statistics are any guide, the market for books that are not even sold in the average bookstore is larger than the market for those that are (...). In other words, the potential book market may be twice as big as it appears to be, if only we can get over the economics of scarcity. Venture capitalist and former music industry consultant Kevin Laws puts it this way: "The biggest money is in the smallest sales."
The Wired article talks about how this phenomenon makes eBay and Amazon much better than traditional books-and-mortar equivalents, who can't afford to stock nearly as many items. What if we take that to the next level? eBay's auction model works for a pretty broad spectrum of person-to-person transactions, maybe broader than any other kind of transaction, but there are surely more transactions that can't fit into eBay's model than those that can.
Imagine a fully decentralized eBay-like marketplace, where eBay would just be the biggest participant among many others; where you have the freedom to build any search facility, any reputation system, any auction terms you like --- and participate seamlessly in transactions with millions of other people, but without having to use eBay's one-size-fits-all terms. eBay could make more money in this marketplace than by going it alone, and many transactions eBay doesn't even consider supporting could take place in the same context.
Posted by kragen at 02:45 PM | Comments (4) | TrackBack
October 06, 2004
blog comment spam
As some readers may have noticed, we've been having a problem with blog comment spam lately --- just like most other Movable Type blogs. We've had several hundred comments, of which only seven have been non-spam. So I've been thinking about countermeasures.
- Soft touch: the more detectable and forceful the anti-spam measures are, the more they will interfere with legitimate users and be circumvented by spammers. Some thoughts:
- Moderate rather than ban: when spam is suspected, mark the comment as "needing approval" rather than rejecting it.
- Different views of the world: when a comment needs approval, show it to its posting IP address as if nothing were wrong. This requires substantially more work for the spammer to detect and circumvent countermeasures.
- Temporary failures: when a spam-appearing comment is posted, hold the connection open for a few seconds, then drop it without sending an HTTP reply. Ordinary people will retry; spamware may or may not.
- IP whitelisting: posts from an IP address that has never had an approved post should be held for moderation.
- Bayesian filtering: simple naive Bayesian classifiers would do a great job of distinguishing spam from nonspam so far.
Posted by kragen at 03:09 PM | Comments (0) | TrackBack
October 05, 2004
TimBL on Semantic Web
Gary Potter posted about an article in MIT’s Magazine of Innovation, Technology Review, in which Mark Frauenfelder of boingboing interviews creator of the World Wide Web Tim Berners-Lee:
On why people aren’t excited about the Semantic Web...Slowly we evolve toward a more understanding, more understandable web...It’s not the first time I’ve had this paradigm-shift problem.On getting past it...
...we are just starting by putting applications onto the Semantic Web one by one and linking them up.... what’s exciting is the network effect.On how the Semantic Web understands data...
Suppose you’re browsing the Web and you find a seminar advertised, and you decide to go. Now, there is all sorts of information on that page, which is accessible to you as a human being, but your computer doesn’t know what it means. So you must open a new calendar entry.....Then get your address book and add new entries for the people involved in the seminar.If there were a Semantic Web version of the page, it would have labeled information on it that would tell the computer “this is an event” and what time and date it is.
If you want to be part of the Semantic Web, you’ll need a friend-of-a-friend file. A FOAF file is a formatted rendition of your personal data – first name, last name, email address, etc. To create the file, you need to fill out a form and an automated process creates the file. Links on how it works and how to implement it on your page are also there.
Posted by adam at 04:55 PM | Comments (0) | TrackBack
October 04, 2004
Halsey's new $50M OnDemand fund for GC apps
This is pretty significant news, as the wheel of creative destructions cycles around yet again -- this new fund certainly is fueled by some of the gains from the success of Salesforce.com. Now, he's in a position to invest in yet further iterations of these ideas: "The fund is specifically for companies who develop services and applications that are developed and delivered via Grand Central's Business Services Network."
Halsey Minor's new fund commits $50M to develop companies on top of Grand Central
Grand Central's Minor announced that he has personally created a $50 million On Demand venture capital fund. Minor established the fund to accelerate the development and deployment of On Demand applications that don't require the use of software or hardware and are delivered as subscription-based services over the Internet. On Demand solutions deliver the same functionality as traditional enterprise software applications but cost significantly less, are less complex to implement and deliver a faster return on investment. Grand Central's new developer program provides the ideal platform for entrepreneurs to build and launch these on demand solutions and then submit them for review and funding. For more details on this announcement see http://www.ondemandventurefund.com/.
Posted by rohit at 02:22 PM | Comments (0) | TrackBack
'Clean Money' with SOAP?
Fascinating gambit; I usually associate it with more mature markets. Perhaps this is a sign that SOAP skills are going to be more profitable for enterprise developers to add to their portfolios soon...
It's also an interesting judging panel -- several of these folks have definitely been around for several hype cycles, so it says something that they're onboard with this bet on WS' maturity.
Grand Central Communications Unveils New Developer Program
"The Golden Spike," Grand Central's first annual contest for developers. The contest starts October 18th in conjunction with the Early Access Program and continues to December 10, 2004, with the winners to be announced in January 2005.
The Golden Spike contest provides developers with an opportunity to showcase their innovative work around reusable business processes and Web services development. Using industry-leading tools and resources provided by Grand Central, participants can submit one or more entries in the following categories:
* Best Business Process
* Best Use of SOAP APIs
* Best Use of Rich Client
The grand prize winner will receive a dream workstation of his or her own creation worth up to $10,000, and there will be three first place winners for each of the categories taking home a $1,000 prize. Contest entries will be judged by a panel of Web services and SOA experts including Tim O'Reilly, founder and chief executive officer of O'Reilly Media; Jason Bloomberg and Ron Schmelzer, senior analysts with industry research firm ZapThink; Bill Appleton, founder, president and chief scientist of DreamFactory; Phil Windley, contributing editor for InfoWorld Test Center; Tony Hong, co-founder of XMethods; Phil Wainewright, chief executive officer, Procullux Ventures, and publisher of Loosely Coupled; and Halsey Minor, chief executive officer, chairman and founder of Grand Central Communications.
Posted by rohit at 02:18 PM | Comments (0) | TrackBack
October 03, 2004
Blogs Are Decentralization Incarnate
From Biz Stone's excellent article, The Wisdom of Blogs:
Bloggers are a wise crowd.
- Diversity of opinion - That's a no-brainer. Bloggers publish hundreds of thousands of posts daily, each one charged with its author's unique opinion.
- Independence of members - Except for your friends saying "You've got to blog about that!" bloggers are not controlled by anyone else.
- Decentralization - There is no central authority in the blogosphere; publish your blog anywhere you want with any tool you want.
- A method for aggregating opinions - Blog feeds make aggregation a snap and there is no shortage of services that take advantage of that fact.
The article goes on to talk about how MIT Media Lab project Blogdex (one of the longest-operating and most-visited opinion aggregators) is like a hive mind of the blogosphere, collectively creating a modern Oracle with no single opinion about anything. Excellent.
Posted by adam at 03:38 PM | Comments (0) | TrackBack
October 02, 2004
Decentralized Filesharing Is Huge
Cachelogic Research paints an interesting picture of decentralized filesharing.
The most astonishing item is that global Internet traffic analysis in June 2004 revealed that in the United States peer-to-peer represents roughly two-thirds of traffic volumes, and in Asia peer-to-peer represents more than four-fifths of traffic volumes. By comparison, HTTP is less than a tenth of the traffic in Asia and less than a sixth of the traffic in the United States. CacheLogic calls peer-to-peer the killer application for broadband with a global reach and a global user base.
Perusing the architectures and protocols section of CacheLogic's site we find a table comparing the characteristics of web traffic (HTTP) with those of common peer-to-peer protocols. They point out that first generation p2p systems were centralized like Napster; second generation p2p systems were decentralized like Gnutella; and now
The third generation architecture is a hybrid of the first two, combining the efficiency and resilience of a centralized network with the stealth characteristics of distributed/decentralised network. This hybrid architecture deploys a hierarchical structure by establishing a backbone network of SuperNodes (or UltraPeers) that take on the characteristics of a central index server. When a client logs on to the network, it makes a direct connection to a single SuperNode which gathers and stores information about peer and content available for sharing.
Recent developments in peer-to-peer include dynamic port selection and bidirectional streaming of download traffic in the most popular peer-to-peer applications in 2004, BitTorrent (more useful thanks to many available BitTorrent clients and DV Guide) and eDonkey (and eMule). BitTorrent is by traffic the most popular peer-to-peer application:
BitTorrent's dominance is likely to be attributed to two factors: the rise in popularity of downloading television programmes, movies and software; and the size of these files - a MP3 maybe 3-5Mb while a BitTorrent often sees files in excess of 500Mb being shared across the Peer-to-Peer network.The high usage of eDonkey in Europe can be attributed to the fact that the eDonkey interface is available in a number of different languages - French, German, Spanish, etc.
So even though the hype machine has stopped pumping p2p, the quieter revolution of the last few years has shown that peer-to-peer traffic has steadily grown to a majority of the Internet traffic worldwide.
Posted by adam at 01:37 PM | Comments (0) | TrackBack
October 01, 2004
Running Reliable Services Is Hard
Phil Windley writes:
Dave Sifry gives some details about the Technorati outage this past weekend. Seems an electrical fire in the data center their co-lo at was the culprit. Running a 24/7 Web application reliably isn't easy and it isn't cheap. It took us several years of problems and study to hit on a solution at iMALL. We finally did figure it out and that was a real lightening of my load. One of the answers is product engineers, an engineer on the operations side whose job it is to make the product (not just the server) work. Properly incented, a product engineer will drive all of the emergency and contingency planning, along with ensuring that engineering delivers a system that can be reliably operated.This just serves as a reminder that it's still hard (and costly) to run a web service reliably.
Posted by adam at 01:46 AM | Comments (0) | TrackBack