실리콘밸리의 비개발자 직장인

certainty equivalent : 확실성등가란 위험있는 수익흐름에 대하여 그 위험을 부담하는 대신 보다 적은 수익이라도 확실하게 실현될 수 있다면 그와 맞바꿀 수 있는 최소한의 가격 (네이버 지식백과) : the amount of money that is equivalent in your mind to a given situation that involves uncertainty. (making hard decision 2nd edi) 예를 들어 이해해 보도록하자, win $2,000 with probability 0.50, lose $20 with probability 0.50 인 복권의 상황을 가정해보자. 만약 친구가 나 대신 위의 복권을 사려고 한다. 얼마에 저 기회를 양보하겠는가? $300..

The easiest way to use a continuous distribution in a decision tree or influence diagram is to approximate it with a discrete distribution. The basic idea is to find a few representative points in the distribution and then to assign those points specific probability value. A particularly simple approach, from Keefer and Bodily (1983), is called the extended Pearson-Turkey method. (Source: Making..

* Data transformation : A function that maps the entire set of values of a given attribute to a new set of replacement values s.t. each old value can be identified with one of the new value - Smoothing : Remove noise from data ex) binning, regressio and clustering - Attribute/feature construction : new attributes constructed from the given ones - Aggregation : Summarization ex) the daily sales d..

* Data integration : Combines data from multiple sources into a coherent dataset - Entity indentification problem : 실제로는 동일한 데이터이나 각각의 소스로부터 다른 방식으로 표현되어 있는 것을 어떻게 합칠 것인가? - 여러 데이터 베이스에서 수집한 자료를 합칠 때 Redundant data 의 문제는 일어나기 마련이다. : Redundant attributes may be able to be detected by correlation analysis and covariance analysis​ - Correlation Analysis (Nominal Data) 1) 카이스퀘어 검정, chi-square test ..

Data preprocessing 1) data cleaning : 누락 데이터 채우기, 노이즈 제거 , 아웃라이어 제거, inconsistency 수정 - Noisy Data : random error or variance in a measured variable => "SMOOTH" by Binning​ First, sort data and partition into (equal-frequency) bins then can smooth by bin means, bin median or bin boundaries : also can smooth by Regression, Clustering, Combined computer and human inspection ex) 다음 데이터(4,8,15,21,21..

FP Growth : A Frequent Pattern- Growth Approach - Divide-and-Conquer Strategy - Avoid explicit candidate generation → Scan only once * FP-tree construction - ex) - 먼저 각각의 item 이, 1-itemset, 의 support count 를 계산하고 가장 높은 item 부터 정렬하는 것이 FP tree 의 작성의 포인트

* Apriori : A candidate generation-and-test approach - method: 1-itemset 을 시작으로 itemset 의 크기를 하나하나 늘여가면서 freqeunt itemset 을 찾아가는 방법 1) Initially, scan DB once to get frequent 1-itemset 2) Generate length (k+1) candidate itemset from length k frequent itemset 3) Test the candidate against the minimum support requirement 4) Terminate when no frequent or candidate set can be generated ex) - Apriori..

- An itemset X is closed frequent if X is frequent and there exists no super pattern Y ⊃ X, with the same support as X - An itemset X is a max frequent if X is frequent and there exists no frequent super patter Y ⊃ X​ - 아래의 표를 바탕으로 생성가능한 itemset 을 표현해보면 1-itemset 부터 5-itemset 까지 만들 수 있다. Total item set의 크기: 2^5-1 minsup = 2 로 설정하여 그래프를 그려보면 아래와 같다. - 위의 node 에서 빨간색으로 표현된 숫자는 Tid 를 나타낸다. 즉, {A} 는..

Frequent Patterns - itemset : A set of one or more items - k-itemset : A itemset contains k items X= {x1, x2, ... , xk} k 개의 아이템을 포함한 아이템셋 - (absolute) support, or support count, or frequency or occurence frequency of an itemset X: the number of transacion that contain the itemset - (relative) Support, s, is the fraction of transactions that contains X (i.e., the probability that a transaction c..

* Minkowski distance i and j are p-dimentional data objects and h is order * Proximity Measure for Ordinal Variable​ - ordinal data: order is important ; rank - can be treated like numeric : replace xif by its rank : map the range of each variable onto [0,1] by replacing i-th object in the f-th attribute by - compute the dissimilarity using methods for numeric attributes -> use euclidean distanc..