Wednesday, September 12, 2007

When algorithms work better than you dream...

So, Timepedia is building a time machine, right? It sounds pretentious, but for us, it's really a geeky moniker of love for our project, after all, is Google's "search engine" really an "engine"? How much horsepower does it have? :)

One part of Timepedia, readers of this blog are already familar with: Chronoscope. With Chronoscope, we are attempting to build an open platform of visualization tools for time oriented data, in much the same way that Google Maps and Google Earth deal with spatial data.

However, what good is a time machine, if you don't know where to go, or don't understand what you're looking at? Timepedia has another platform, aimed at data mining time related information, called Everett (owned and implemented by another Timepedia founder, Mat). Everett is a collection of many algorithms for both data mining, and forecasting, some of them bleeding edge academic research. When we started, we weren't sure which of them would work, or how well they would work, we only knew that they had promising features, so Everett was less of a end user product, and more of a research platform.

One of the tools of Everett is an algorithm that lets us find hidden recurring patterns in data, even in the presence of noise, or scaling. Last week, we tested the algorithm on real life data for the first time, and had one of those "holy cow!" moments, which don't occur too often for me personally, where your own code surprises you.

To give you an example, I fed Everett an 18,000 data point series of federal funds rates over the last few decades, and it identified a pattern that occured 3 times in history. Visualizing this in another tool we call Timelord (A Chronoscope married to Everett and other server-side services), I was puzzled as to the significance of these three sequences. My co-founder Shawn spent about 1 hour Googling, until he found the correlation: These sequences corresponded to international financial/currency crises (such as the Mexican currency crises), in which the Fed was forced to take action. The leadup to the crises appeared identical each time. A fluke? It sure the hell was very interesting.

I was worried it was a fluke, so I tried something more mundane. A time series of unemployment benefit expenditures in Indiana, and once again, Everett identified a series of puzzling repetitive sequences. What were they? The dates looked very familar, 1980-81, 1990-91, 2000-1...were they recessions? To check, I used Timelord to overlay a National Bureau of Economic Research official measure of economic expansions and contractions, and sure enough, these patterns intersected with NBER recessions. One other interesting property stood out, the patterns returned prefixed the recessions, that is, Everett was showing us a pattern that leads to a recession.

How cool is that? Ambition got the best of me, I went for broke: I tried a historical time series of average hurricane strength (saffir-simpson scale), as well as a yearly count. There appears to be good evidence that a 40-60 cyclical hurricane season exists, and I was hoping that Everett could find these patterns, but alas, it did not.

Still, the initial results are promising, and we hope that Everett will give average users an ability to query time in ways that have not been previously available.

So, if you're wondering why I haven't released Chronoscope yet, it's because I've been working on integrating Timelord with Everett. :)

p.s. Timelord is another GWT application, making it our 4th major GWT application. Everett is C++ coupled via JNI a Java/GWT RPC interface, since performance is absolutely critical in Everett.


Cory said...

Have there been any papers been published about the Everett pattern finding algorithm?

It sounds fascinating.

Ray Cromwell said...

The algorithm being used isn't ours, it was published in the last few years, and presented at KDD'07. I hope I didn't convey the impression that we had invented it (we've only tweaked some of its parameters), since that would be a serious disservice to its inventor, who I think deserves credit for blazing a revolutionary trail in time series analysis. We will fully disclosee all of the relavent citations and the inner workings of Everett when the Timepedia launches. -Ray

Laszlo said...

What you mention is pretty standard stuff in Data Mining, it's 'called frequent episode discovery', it's used in all kinds of industrial monitoring systems, and in telecom.

Catalin I said...

Can I use this algorithm to predict the numbers at lottery ? :)

Ray Cromwell said...

We are not using FED, we are using a new motif discovery algorithm. FED is far far too slow for our purposes, we need to detect motifs in 10k or 100k long datasets with millisecond response times and oodles of concurrent web users.

We're also not interested in the rules output that FED gives you, instead, we overlay historical timelines and news feeds that best fit discovered Motifs so that human beings can make a judgement call, and write an interpretation.

sexy said...




色情A片,A片下載,色情遊戲,色情影片,色情聊天室,情色電影,免費視訊,免費視訊聊天,免費視訊聊天室,一葉情貼圖片區,情色視訊,免費成人影片,視訊交友,視訊聊天,言情小說,愛情小說,AV片,A漫,av dvd,情色論壇,視訊美女,AV成人網,情色文學,成人交友,成人電影,成人貼圖,成人小說,成人文章,成人圖片區,成人遊戲,愛情公寓,情色貼圖,成人論壇,色情


信次 said...

情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,美國aneros,rudeboy,英國rudeboy,英國Rocksoff,德國Fun Factory,Fun Factory,英國甜筒造型按摩座,甜筒造型按摩座,英國Rock Chic ,瑞典 Lelo ,英國Emotional Bliss,英國 E.B,荷蘭 Natural Contours,荷蘭 N C,美國 OhMiBod,美國 OMB,Naughti Nano ,音樂按摩棒,ipod按摩棒,美國 The Screaming O,美國TSO,美國TOPCO,美國Doc Johnson,美國CA Exotic,美國CEN,美國Nasstoy,美國Tonguejoy,英國Je Joue,美國Pipe Dream,美國California Exotic,美國NassToys,美國Vibropod,美國Penthouse,仿真按摩棒,矽膠按摩棒,猛男倒模,真人倒模,仿真倒模,PJUR,Zestra,適趣液,穿戴套具,日本NPG,雙頭龍,FANCARNAL,日本NIPPORI,日本GEL,日本Aqua Style,美國WET,費洛蒙,費洛蒙香水,仿真名器,av女優,打炮,做愛,性愛,口交,吹喇叭,肛交,魔女訓練大師,無線跳蛋,有線跳蛋,震動棒,震動保險套,震動套,TOY-情趣用品,情趣用品網,情趣購物網,成人用品網,情趣用品討論,成人購物網,鎖精套,鎖精環,持久環,持久套,拉珠,逼真按摩棒,名器,超名器,逼真老二,電動自慰,自慰,打手槍,仿真女郎,SM道具,SM,性感內褲,仿真按摩棒,pornograph,hunter系列,h動畫,成人動畫,成人卡通,情色動畫,情色卡通,色情動畫,色情卡通,無修正,禁斷,人妻,極悪調教,姦淫,近親相姦,顏射,盜攝,偷拍,本土自拍,素人自拍,公園露出,街道露出,野外露出,誘姦,迷姦,輪姦,凌辱,痴漢,痴女,素人娘,中出,巨乳,調教,潮吹,av,a片,成人影片,成人影音,線上影片,成人光碟,成人無碼,成人dvd,情色影音,情色影片,情色dvd,情色光碟,航空版,薄碼,色情dvd,色情影音,色情光碟,線上A片,免費A片,A片下載,成人電影,色情電影,TOKYO HOT,SKY ANGEL,一本道,SOD,S1,ALICE JAPAN,皇冠系列,老虎系列,東京熱,亞熱,武士系列,新潮館,情趣用品,情趣,情趣商品,情趣網站,跳蛋,按摩棒,充氣娃娃,自慰套,G點,性感內衣,情趣內衣,角色扮演,生日禮物,生日精品,自慰,打手槍,潮吹,高潮,後庭,情色論譠,影片下載,遊戲下載,手機鈴聲,音樂下載,開獎號碼,統一發票號碼,夜市,統一發票對獎,保險套,做愛,減肥,美容,瘦身,當舖,軟體下載,汽車,機車,手機,來電答鈴,週年慶,美食,徵信社,網頁設計,網站設計,室內設計,靈異照片,同志,聊天室,運動彩券,大樂透,威力彩,搬家公司,除蟲,偷拍,自拍,無名破解,av女優,小說,民宿,大樂透開獎號碼,大樂透中獎號碼,威力彩開獎號碼,討論區,痴漢,懷孕,美女交友,交友,日本av,日本,機票,香水,股市,股市行情, 股市分析,租房子,成人影片,免費影片,醫學美容,免費算命,算命,姓名配對,姓名學,姓名學免費,遊戲,好玩遊戲,好玩遊戲區,線上遊戲,新遊戲,漫畫,線上漫畫,動畫,成人圖片,桌布,桌布下載,電視節目表,線上電視,線上a片,線上掃毒,線上翻譯,購物車,身分證製造機,身分證產生器,手機,二手車,中古車,法拍屋,歌詞,音樂,音樂網,火車,房屋,情趣用品,情趣,情趣商品,情趣網站,跳蛋,按摩棒,充氣娃娃,自慰套, G點,性感內衣,情趣內衣,角色扮演,生日禮物,精品,禮品,自慰,打手槍,潮吹,高潮,後庭,情色論譠,影片下載,遊戲下載,手機鈴聲,音樂下載,開獎號碼,統一發票,夜市,保險套,做愛,減肥,美容,瘦身,當舖,軟體下載,汽車,機車,手機,來電答鈴,週年慶,美食,徵信社,網頁設計,網站設計,室內設計,靈異照片,同志,聊天室,運動彩券,,大樂透,威力彩,搬家公司,除蟲,偷拍,自拍,無名破解, av女優,小說,民宿,大樂透開獎號碼,大樂透中獎號碼,威力彩開獎號碼,討論區,痴漢,懷孕,美女交友,交友,日本av ,日本,機票,香水,股市,股市行情,股市分析,租房子,成人影片,免費影片,醫學美容,免費算命,算命,姓名配對,姓名學,姓名學免費,遊戲,好玩遊戲,好玩遊戲區,線上遊戲,新遊戲,漫畫,線上漫畫,動畫,成人圖片,桌布,桌布下載,電視節目表,線上電視,線上a片,線上a片,線上翻譯,購物車,身分證製造機,身分證產生器,手機,二手車,中古車,法拍屋,歌詞,音樂,音樂網,借錢,房屋,街頭籃球,找工作,旅行社,六合彩,整型,整型,珠海,雷射溶脂,婚紗,網頁設計,水噹噹,台中隆鼻,果凍隆乳,改運整型,自體脂肪移植,新娘造型,婚禮顧問,下川島,常平,常平,珠海,澳門機票,香港機票,貸款,貸款,信用貸款,宜蘭民宿,花蓮民宿,未婚聯誼,網路購物,婚友,婚友社,未婚聯誼,交友,婚友,婚友社,單身聯誼,未婚聯誼,未婚聯誼, 婚友社,婚友,婚友社,單身聯誼,婚友,未婚聯誼,婚友社,未婚聯誼,單身聯誼,單身聯誼,白蟻,白蟻,除蟲,老鼠,減肥,減肥,在家工作,在家工作,婚友,單身聯誼,未婚聯誼,婚友,交友,交友,婚友社,婚友社,婚友社,大陸新娘,大陸新娘,越南新娘,越南新娘,外籍新娘,外籍新娘,台中坐月子中心,搬家公司,搬家公司,中和搬家,台北搬家,板橋搬家,新店搬家,線上客服,網頁設計,線上客服,網頁設計,植牙,關鍵字,關鍵字,seo,seo,網路排名,自然排序,網路排名軟體,交友,越南新娘,婚友社,外籍新娘,大陸新娘,越南新娘,交友,外籍新娘,視訊聊天,大陸新娘,婚友社,婚友,越南新娘,大陸新娘,越南新娘,視訊交友,外籍新娘,網路排名,網路排名軟體,網站排名優化大師,關鍵字排名大師,網站排名seo大師,關鍵字行銷專家,關鍵字,seo,關鍵字行銷,網頁排序,網頁排名,關鍵字大師,seo大,自然排名,網站排序,網路行銷創業,汽車借款,汽車借錢,汽車貸款,汽車貸款,拉皮,抽脂,近視雷射,隆乳,隆鼻,變性,雙眼皮,眼袋,牙齒,下巴,植牙,人工植牙,植髮,雷射美容,膠原蛋白,皮膚科,醫學美容,玻尿酸,肉毒桿菌,微晶瓷,電波拉皮,脈衝光,關鍵字,關鍵字,seo,seo,網路排名,自然排序,網路排名軟體,汽車借款,汽車借款,汽車借款,汽車貸款,汽車貸款,借錢,借貸,當舖,借款,借貸,借錢,週轉,學英文,英文社團,英語俱樂部,學習英文,英語會話,英文演講,English Club,學英語,學英文,美語社團,英語社團,英文讀書會,Toastmasters,Toastmaster,英語讀書會,拍樂得批發,拍樂得飾品,拍樂得化妝品批發,

平平 said...

^^ nice blog!! ^@^

徵信, 徵信網, 徵信社, 徵信社, 徵信社, 徵信社, 感情挽回, 婚姻挽回, 挽回婚姻, 挽回感情, 徵信, 徵信社, 徵信, 徵信, 捉姦, 徵信公司, 通姦, 通姦罪, 抓姦, 抓猴, 捉猴, 捉姦, 監聽, 調查跟蹤, 反跟蹤, 外遇問題, 徵信, 捉姦, 女人徵信, 女子徵信, 外遇問題, 女子徵信, 徵信社, 外遇, 徵信公司, 徵信網, 外遇蒐證, 抓姦, 抓猴, 捉猴, 調查跟蹤, 反跟蹤, 感情挽回, 挽回感情, 婚姻挽回, 挽回婚姻, 外遇沖開, 抓姦, 女子徵信, 外遇蒐證, 外遇, 通姦, 通姦罪, 贍養費, 徵信, 徵信社, 抓姦, 徵信社, 徵信, 徵信公司, 徵信社, 徵信, 徵信公司, 徵信社, 徵信公司, 女人徵信, 外遇

徵信, 徵信網, 徵信社, 徵信網, 外遇, 徵信, 徵信社, 抓姦, 徵信, 女人徵信, 徵信社, 女人徵信社, 外遇, 抓姦, 徵信公司, 徵信社, 徵信社, 徵信社, 徵信社, 徵信社, 徵信社, 女人徵信社, 徵信社, 徵信, 徵信社, 徵信, 女子徵信社, 女子徵信社, 女子徵信社, 女子徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信社,

徵信, 徵信社,徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 外遇, 抓姦, 離婚, 外遇,離婚,

徵信, 外遇, 離婚, 徵信社, 徵信, 外遇, 抓姦, 徵信社, 徵信, 徵信社, 徵信, 外遇, 徵信社, 徵信, 外遇, 抓姦, 徵信社, 征信, 征信, 徵信, 徵信社, 徵信, 徵信社, 征信, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信社, 徵信社, 徵信, 外遇, 抓姦, 徵信, 徵信社, 徵信, 徵信社, 徵信,