Background of the Project

At the time of this experiment, Udacity courses currently have two options on the course overview page: "start free trial", and "access course materials". If the student clicks "start free trial", they will be asked to enter their credit card information, and then they will be enrolled in a free trial for the paid version of the course. After 14 days, they will automatically be charged unless they cancel first. If the student clicks "access course materials", they will be able to view the videos and take the quizzes for free, but they will not receive coaching support or a verified certificate, and they will not submit their final project for feedback.
In the experiment, Udacity tested a change where if the student clicked "start free trial", they were asked how much time they had available to devote to the course. If the student indicated 5 or more hours per week, they would be taken through the checkout process as usual. If they indicated fewer than 5 hours per week, a message would appear indicating that Udacity courses usually require a greater time commitment for successful completion, and suggesting that the student might like to access the course materials for free. At this point, the student would have the option to continue enrolling in the free trial, or access the course materials for free instead. This screenshot shows what the experiment looks like.
The hypothesis was that this might set clearer expectations for students upfront, thus reducing the number of frustrated students who left the free trial because they didn't have enough time—without significantly reducing the number of students to continue past the free trial and eventually complete the course. If this hypothesis held true, Udacity could improve the overall student experience and improve coaches' capacity to support students who are likely to complete the course.
The unit of diversion is a cookie, although if the student enrolls in the free trial, they are tracked by user-id from that point forward. The same user-id cannot enroll in the free trial twice. For users that do not enroll, their user-id is not tracked in the experiment, even if they were signed in when they visited the course overview page.
Cited from Udacity

Metric Choice

Invariant Metric

1. Number of Cookies: That is, number of unique cookies to view the course overview page.

Reason: “In the experiment, Udacity tested a change where if the student clicked ‘start free trial’, they were asked how much time they had available to devote to the course.” Data of number of cookies to view the course overview page is before the change. Therefore, number of cookies should not be influenced by the experiment. It should be an invariant metric.

1. Number of Clicks: That is, number of unique cookies to click the "Start free trial" button (which happens before the free trial screener is trigger).

Reason: The behavior of clicking the "Start free trial" button is before the checking out process which is the experiment. Therefore number of clicks should not influenced by the experiment. It should be an invariant metric.

1. Click Through Probability: That is, number of unique cookies to click the "Start free trial" button divided by number of unique cookies to view the course overview page.

Reason: Both number of unique cookies to click the "Start free trial" button and number of unique cookies to view the course overview page are invariant metrics. Therefore click through probability which is unique cookies to click the "Start free trial" button divided by number of unique cookies to view the course overview page is an invariance metric.

Evaluation Metric

1. Gross Conversion: That is, number of user-ids to complete checkout and enroll in the free trial divided by number of unique cookies to click the "Start free trial" button.

Reason: Gross conversion are number of user-ids to complete checkout and enroll in the free trial divided by number of unique cookies to click the "Start free trial" button. In the experiment, users will be asked an question of validate study time, and who choose the option “fewer than 5 hours per week” will be given another suggestion to access to “course materials for free”. It will make an influence on number of user-ids enrolling in the free trial by setting clearer expectations for students upfront, thus reducing the number of frustrated students who left the free trial because they didn't have enough time, which is the hypothesis of the experiment. Addition , number of unique cookies to click the "Start free trial" button are invariant metric and the unit of diversion is a cookie. Therefore, gross conversion is chosen as an evaluation metric.

1. Net Conversion: That is, number of user-ids to remain enrolled past the 14-day boundary (and thus make at least one payment) divided by the number of unique cookies to click the "Start free trial" button.

Reason: Number of user-ids who enroll in the free trial will be influenced by the checkout process in the experiment. Remaining enrolled and making at least one payment is the process after enrolling in the free trial. Therefore, number of user-ids could be influenced by the experiment. Gathering data of number of user-ids to remain enrolled will help to test the hypothesis that “without significantly reducing the number of students to continue past the free trial”. Addition , number of unique cookies to click the "Start free trial" button is invariant metric and the unit of diversion is a cookie . Therefore, net conversion is chosen as an evaluation metric.

Metric which are not chosen as evaluation or invariant metric.

1. Number of user-ids: That is, number of users who enroll in the free trial.

Reason: Number of user-ids will be affected by the experiment. However, the difference between number of user-ids in the control group and the experiment group may be caused by experiment or data gathering. Therefore, using conversion is a better way to make comparison between the control group and the experiment group.

1. Retention: That is, number of user-ids to remain enrolled past the 14-day boundary (and thus make at least one payment) divided by number of user-ids to complete checkout.

Reason: After sample size calculation, more than 100 thousand pageviews will be needed, which can be hardly gathering in the experiment. Therefore retention will not be used in the experiment.

The hypothesis of the experiment is that set clearer expectations for students upfront, thus reducing the number of frustrated students who left the free trial because they didn't have enough time—without significantly reducing the number of students to continue past the free trial and eventually complete the course. Therefore, the result that is expected is that gross conversion is significantly changed in the experiment group. The net conversion is not significantly changed in the experiment at the same time.

Measuring Standard Deviation

Standard Deviation of Gross Conversion: 0.0202

Standard Deviation of Net Conversion: 0.0156

Because in both gross conversion and net conversion the unit of analysis are cookies. And the unit of diversion. The unit of diversion and unit of analysis are match. Therefor, there is no need to calculate empirical variability.

Sizing

Not using Bonferroni correction. Pageviews: 685325

Duration vs. Exposure

Fraction of Traffic Expose: 0.62 Length of Experiment: 28 days

Considering seasoning factor which is unique cookies to view course overview page may have difference between weekday and weekends. Therefore 28days (4 weeks) is chosen as the length of the experiment. Although more than half of the traffic will be in the experiment. In order not to expand the experiment too long, the fraction of 0.62 are chosen.

Students in the experiment would not take risks also for four reasons below: 1. Students are allowed to access to the free trial even their study time is less than 5 hours per week. 2. Process of enrollment or taking free material are almost the same in the experiment, only a checkout will be added. Student would not have to spent much time to get adapted to the checkout in the experiment. 3. The experiment would not change the database. 4. Students’ private information would not be gathering during the checkout.

Experiment Analysis

Sanity Checks

Number of Cookies 95% CI = （0.4988， 0.5011） Observed Value: 0.5006 Pass the sanity check.

Number of Clicks 95% CI = (0.4958, 0.5041) Observed Value: 0.5004 Pass the sanity check.

Click Through Probability 95% CI = (-0.0012, 0.0013) Observed Value: 0.000 Pass the sanity check.

Result Analysis

Effect Size Tests

Gross Conversion

Null hypothesis (Ho):
Number of user-ids who enroll in the free trial is not changed by the experiment. And gross conversion in the control group and the experiment group are almost the same (d=0).

Alternative Hypothesis (HA): Number of user-ids who enroll in the free trial is significantly changed by the experiment. And gross conversion in the control group and the experiment group are significantly different and not by chance (d≠0).

two trail test α = 0.05

SEpool = 0.0034 d^ = -0.0049 dmin= 0.01 95% CI = (-0.0291, -0.012)

Result: reject Ho Number of user-ids who enroll in the free trial is significantly changed by the experiment. Gross conversion in experiment group is statistical significant and practically significant changed compared to control group.

Net Conversion

Null hypothesis (Ho):
Number of user-ids to remain enrolled past the 14-day boundary is significantly changed in the experiment. Net conversion in the control group and the experiment group are significantly different and not by chance (d≠0).

Alternative Hypothesis (HA):
Number of user-ids to remain enrolled past the 14-day boundary is not changed in the experiment. And net conversion in the control group and the experiment group are almost the same (d=0).

two trail test α = 0.05

SEpool = 0.0034 d^ = -0.0049 dmin = 0.0075 95% CI = (-0.0116, 0.0019)

Result: reject Ho Net conversion in experiment group is neither statistical significant nor practical significant changed compared to control group.

Sign Tests

Gross Conversion

Null hypothesis (Ho):
Number of user-ids who enroll in the free trial is not changed by the experiment. And gross conversion in the control group and the experiment group are almost the same.

Alternative Hypothesis (HA): Number of user-ids who enroll in the free trial is significantly changed by the experiment. And gross conversion in the control group and the experiment group are significantly different and not by chance.

two trail test α = 0.05

P value = 0.0026

Result: Reject Ho Number of user-ids who enroll in the free trial is significantly changed by the experiment. Gross conversion in experiment group is statistical significant changed compared to control group.

Net Conversion

Null hypothesis (Ho):
Number of user-ids to remain enrolled past the 14-day boundary is significantly changed in the experiment. Net conversion in the control group and the experiment group are significantly different and not by chance.

Alternative Hypothesis (HA):
Number of user-ids to remain enrolled past the 14-day boundary is not changed in the experiment. And net conversion in the control group and the experiment group are almost the same.

two trail test α = 0.05

P value = 0.6776

Result: Reject Ho Number of user-ids to remain enrolled past the 14-day boundary is not significantly changed in the experiment. Net conversion in experiment group is not statistical significant changed compared to control group.

Summary

Bonferroni correction is not used in the analysis.

In order to use Bonferroni correction, αoverall has be to calculated, which is assuming that metrics are independent. However in the experiment metric are related and will move together. Bonferroni correction is way much conservative for the experiment.

Recommendation

I will not launch the experiment. Even thought the gross conversion is significantly improved in the experiment, the confidence interval of net conversion are between -0.0116 and 0.0019,. The experiment may cause the decrease of net conversion. The P value of net conversion is 0.6676, the net conversion could practically significantly decrease to some degree.

Follow-Up Experiment

Student cancellation early in the course is caused by a variety of reasons, like lacking of study time, or not meet the knowledge requirements for taking the course. The experiment I will try is that when students clicked "start free trial” button, they will be asked to take a test about the required knowledge for taking this course. By evaluating the scores of the test, it will give student suggestion if they need take other courses to meet the requirement.

My hypothesis would be this experiment will reduce the student who left the free trial because they didn’t meet the knowledge requirement of the course without significantly reducing the number of students to continue past the free trial.

The unit of diversion is a cookie.

Metric: 1. Number of Cookies: That is, number of unique cookies to view the course overview page.

Reason: Invariant metric, for sanity check.

1. Number of Clicks: That is, number of unique cookies to click the "Start free trial" button.

Reason: Invariant metric, for sanity check.

1. Click Through Probability: That is, number of unique cookies to click the "Start free trial" button divided by number of unique cookies to view the course overview page.

Reason: Invariant metric, for sanity check.

1. Gross Conversion: That is, number of user-ids to complete test and enroll in the free trial divided by number of unique cookies to click the "Start free trial" button. Reason: Evaluation metric, to evaluate whether test will effect students choice to enroll in the free trial.

2. Net Conversion: That is, number of user-ids to remain enrolled past the 14-day boundary (and thus make at least one payment) divided by the number of unique cookies to click the "Start free trial" button.

Reason: Evaluation metric, to evaluate whether test will effect number of students to remain enrolled past the 14-day boundary.

1.描述统计学入门

2.推论统计学入门

3.数据分析入门（数据分析使用的工具：pands和numpy，需要有一般的python基础）

4.使用 Tableau 进行数据可视化

5.A/B 测试

圣地中的佛教圣地——高野山。唐朝时期，日本僧人空海，谥号弘法大师，在唐朝学习当时较为流行的密宗佛教，回到日本后，受到皇家的重视，并拨予了高野山给他，开办真言宗修行场所。真言宗的总寺庙是高野山的金刚峰寺。

熊野三山的熊野本宫大社、熊野那智大社、熊野速玉大社，则是神道教圣地。

而吉野山也是自7世纪以来，聚集了众多的修行者。

历代的天皇和将军都会来这些地方参拜或赏樱，甚至埋葬于此。

朝圣路包括： 熊野参詣道: 大边路，中边路，小边路、伊势路 高野参詣道: 町石道、三谷坂、京大坂道不動坂、黒河道、女人道

（图片由原始上傳者為日语维基百科的Yosemite - 本檔案是從ja.wikipedia轉移到維基共享資源。公有领域，原图片链接）

在这些世界遗产中，相对比较热门也方便到达的地方有：高野山（可乘坐南海电铁到达），熊野本宫大社（可乘坐巴士到达），熊野那智大社和熊野速玉大社（可乘坐巴士到达）

日本和歌山旅游局有关于朝圣路非常详尽的介绍，包括徒步地图，提供住宿建议地点，并已经合理的把徒步时间划分好，并给出大概的徒步时间。这个时间按照徒步的起始和终点，已经徒步的方向来计算，都是相对来说准确的。（亲测）

徒步道上会有非常清晰的路牌。官方还提供盖章本，和特定的盖章地点，完成某些条件，就可以获得【XX踏破証明書】，在下面的章节里会详细提到。

虽然官方资料都是日文的，当时能看个大概意思并不困难，也不影响使用。以下内容会主要引用和歌山旅游局网站，做一个信息的梳理，并给出原文的链接供大家查找和制作攻略。

徒步能体验到什么？ 1.森林，杉树都非常高，有雾气的时候会有点灵异 2.日本的乡下，大白天横穿乡村都看不到一个人 3.佛教文化，高野山的寺庙还提供住宿（宿坊），包含早餐和晚餐的精进料理（素食），可以体验早上僧人的早课和仪式。 4.神道教文化 5.温泉，一路上有几个温泉乡

高野参詣道

高野参詣道—— 町石道

（高野参詣道｜街道マップ｜和歌山県観光情報）

1.九度山駅～上古沢駅（九度山駅（九度山町）～上古沢駅（九度山町））

原地图见和歌山旅游局

步行距离: 13.1公里 标准步行时间4小时06分钟 标准花费时间6小时15分钟（包括休息啊，某些观光台可以需要走岔路等）

2.上古沢駅～壇上伽藍 (上古沢駅（九度山町）～壇上伽藍（高野町）)

原地图见和歌山旅游局

步行距离：16.2公里 标准步行时间5小时10分钟 标准花费时间6小时55分钟

3.壇上伽藍～大師御廟（高野山奥之院内）（壇上伽藍（高野町）～大師御廟（高野町））

原地图见和歌山旅游局

步行距离：3.4公里 标准步行时间56分钟 标准花费时间2小时02分钟

这条路基本贯穿了高野山的主要景点和寺庙，终点是位于奥之院尽头的弘法大师御庙。

高野参詣道——三谷坂

妙寺駅～丹生都比売神社（妙寺駅～丹生都比売神社（かつらぎ町））

原地图见和歌山旅游局

步行距离：7.5公里 标准步行时间2小时16分钟 标准花费时间3小时

高野三山——女人道

高野三山巡り・女人道巡り（https://www.wakayama-kanko.or.jp/walk/koya/kouyasanzanmeguri.html）

原地图见和歌山旅游局

步行距离：10.7公里 标准步行时间2小时17分钟 标准花费时间3小时35分钟

古时候的日本，对于修行之地，有专门针对女性的禁忌（日语：女人禁制），有在特定仪式或者特定时间不允许女性进入的，也有永久不允许女性进入的，所以高野山有专门给女性设置的道路，叫做“女人道”。不过1904年，也就是明治37年，高野山就解禁了。

高野七口——京大坂道不動坂、黒河道、大峰道、有田・龍神道

（高野参詣道｜街道マップ｜和歌山県観光情報）

京大坂道不動坂 学文路駅——不動坂女人堂（高野町） 原地图见和歌山旅游局

步行距离：16.2公里 标准步行时间5小时10分钟 标准花费时间6小时55分钟

黒河道 橋本駅——高野幹部交番 原地图见和歌山旅游局

步行距离：18.1公里 标准步行时间5小时44分钟 标准花费时间7小时35分钟

大峰道 小代下（五條市）——奥之院（高野町）

原地图见和歌山旅游局

步行距离：16.9公里 标准步行时间4小时39分钟 标准花费时间6小时15分钟

有田・龍神道 はなぞの温泉花圃の里～大門 神龍口

原地图见和歌山旅游局

步行距离：12.7公里 标准步行时间4小时20分钟 标准花费时间5小时05分钟

高野参詣道汇总

来自和歌山旅游局

熊野参詣道

熊野参拜道小辺路

（https://www.wakayama-kanko.or.jp/walk/kohechi/index.html）

从高野山一直走到熊野本宫大社，推荐的徒步时间是4天3晚。和歌山旅游局网站上的线路不太标准。推荐使用奈良县十津川村的指南【十津川村観光協会】　世界遺産熊野参詣道登山マップ。纸质版可以在高野山的游客中心拿到，没有摆放出来，需要问工作人员要。

4日徒步 高野山〜大股，大股〜三浦口，三浦口〜十津川温泉，十津川温泉〜熊野本宮大社

第一天 高野山〜大股

步行距离：16.8公里 标准步行时间5小时 标准花费时间6小时40分钟

第二天 大股〜三浦口

步行距离：15.9公里 标准步行时间5小时 标准花费时间6小时30分钟

第三天 三浦口〜十津川温泉

步行距离：19.2公里 标准步行时间6小时 标准花费时间7小时50分钟

第四天（官方地图是分为 ：十津川温泉〜八木尾，八木尾〜熊野本宮大社，但是因为八木尾〜熊野本宮大社，只有2个小时，故合并） 十津川温泉〜八木尾〜熊野本宮大社

步行距离：15公里 标准步行时间5小时30分钟 标准花费时间7小时30分钟

住宿预订： 小边路上基本都是民宿和温泉旅馆。快速预订熊野古道上的民宿，请认准田辺市熊野ツーリズムビューロー |，可以按照所走道路，选择合适的村落，筛选住宿。住宿基本是半食宿的，包含晚饭和第二天早饭，还可以预订中午的便当。不过预订后，需要3天进行确认。

交通 高野山是热门旅游景点，可以乘坐南海电铁到达，高野山内的林间巴士，也为南海电铁所有，可以根据需求，购买通票。

从高野山去往大股是没有常规巴士的，只有在热门季节需要提前两日预约的南海林间巴士。当然，还有一种办法是taxi去，土豪可以参考，基本需要1000元人民币。

不过从十津川温泉到熊野本宫大社，可以乘坐奈良巴士，线路和时间表奈良交通バス案内システム

熊野古道基本会遇到的几个巴士公司： 1.龙神巴士（龍神バス），巴士线路图(貸切バス・路線バスの龍神自動車|路線バス|運行系統図 2.明光巴士（明光バス），熊野古道巴士线路熊野古道へのバス | 明光バス株式会社 3.南海巴士（南海バス） 4.奈良巴士（奈良バス），奈良交通

所以

熊野古道 中辺路

(https://www.wakayama-kanko.or.jp/walk/nakahechi/index.html)

熊野古道 大辺路

（https://www.wakayama-kanko.or.jp/walk/oohechi/index.html）

熊野古道盖章本购买地

《圣地牙哥巡礼和熊野古道巡礼》 盖章本：

1. 熊野古道館 和歌山県田辺市中辺路町栗栖川１２２２−１ (滝尻王子 附近）可购买中边路盖章本（100日元）

2. 世界遺産 熊野本宮館内 位于熊野本宫大社

3. 可购买中边路盖章本（100日元）

4. 新宮市観光協会 和歌山県新宮市徐福2-1-1　新宮駅構内 可购买中边路盖章本（100日元）

5. 那智勝浦町観光協会 〒649-5335 和歌山県東牟婁郡那智勝浦町築地6丁目1番地1 可购买中边路盖章本（100日元）

6. 田辺市観光センター 〒646-0031　和歌山県田辺市湊1番20号

7. 一般社団法人高野山宿坊協会 和歌山県伊都郡高野町高野山600番地 （奥之院附近）

8. わかやま紀州館 東京都千代田区有楽町2-10-1 東京交通会館地下1階

高野七口、大边路、纪伊路盖章本可以免费下载

熊野古道专门的盖章本有4本（来源) 1.熊野古道中辺路押印帳 需要100日元购买 36个章 2.高野七口 18个章 3.熊野古道紀伊路 21个章 4.熊野古道大辺路［富田坂・仏坂・長井坂］ 6个章

完成1，可以获得【中辺路オリジナルの証明書】 完成2、3，可以获得【高野七口・熊野古道紀伊路の踏破証明書】 4个全部完成，可以获得“和歌山四参詣道完全踏破証明書”

Background Information

In a Stroop task, participants are presented with a list of words, with each word displayed in a color of ink. The participant’s task is >to say out loud the color of the ink in which the word is printed. The task has two conditions: a congruent words condition, and an >incongruent words condition. In the congruent words condition, the words being displayed are color words whose names match the colors in >which they are printed: for example RED, BLUE. In the incongruent words condition, the words displayed are color words whose names do not >match the colors in which they are printed: for example PURPLE, ORANGE. In each case, we measure the time it takes to name the ink colors >in equally-sized lists. Each participant will go through and record a time from each condition.

Dataset from Udacity

Distribution of time taken in congruent group

Participants who spent between 12 and 13 are the most.

Distribution of time taken in incongruent group

Participants who spent between 17 and 18, 20 and 21 are the most.

Distribution of time taken in both the congruent group and the incongruent group.

Every participant spends more time in the incongruent condition than that in the congruent group.

Dependent T-test for Paired Samples

Why is a t-test chosen?

1.The sample size is below 30.

2.The population standard deviation is unknown.

Why is a dependent test chosen?

In the Stroop task, each participant will go through and record a time from each condition.

Independent Variable:

a congruent words condition, and an incongruent words condition.

Dependent Variable:

the time it takes to name the ink colors in equally-sized lists in a congruent words condition and an incongruent words condition.

Null hypothesis (Ho):

The mean of time taken in congruent condition and that in incongruent condition of the population are almost equal. The mean of time taken in both condition of the population are almost equal (μc-μI = 0). And the difference of the population mean is equal to zero (μD = 0).

Alternative Hypothesis (HA):

The mean of time taken in both condition of the population are significantly different (μc-μI ≠ 0) and not by chance. And the difference of population mean is not equal to zero (μD ≠ 0).

Hypothesis:

Ho: μD = 0

HA: μD ≠ 0

two trail test

α = 0.05

df = 23

t critical value = ±2.069

mean of Congruent Group = 14.05

mean of Incongruent Group = 22.02

SD of Difference = 4.86

t-statistics = -8.02

p<.05

Confidence interval on the mean difference; 95% CI = （-10.02， 5.91）

d = 1.64

Result: reject Ho

Because the probability of getting t-statistics which is -8.03 is less than 5%. Time taken in congruent group is statistical significantly less than that in Incongruent group.