如果你对降采样还不是很熟的话，请阅读之前的文章 "Elasticsearch：对时间序列数据流进行降采样（downsampling)"。这是一个简化的示例，可让你快速了解降采样如何作为 ILM 策略的一部分来减少一组采样指标的存储大小。 该示例使用典型的 Kubernetes 集群监控数据。 要使用 ILM 测试降采样，请执行以下步骤：
- 检查先决条件。
- 创建索引生命周期策略。
- 创建索引模板。
- 摄取时间序列数据。
- 查看结果。
以下展示是使用最新的 Elastic Stack 8.11 来进行的。
先决条件
- 请参阅时间序列数据流先决条件。
- 集群权限：manage_ilm 和 manage_index_templates。
- 索引权限：你创建或转换的任何 TSDS 的 create_doc 和 create_index。 要滚动 TSDS，你必须具有 manage 权限。
- 在运行此示例之前，你可能需要尝试手动运行降采样示例。
创建索引生命周期策略
为你的时间序列数据创建 ILM 策略。 虽然不是必需的，但建议使用 ILM 策略来自动管理时间序列数据流索引。
要启用降采样，请添加 Downsample 操作并将 fixed_interval 设置为要聚合原始时间序列数据的降采样间隔。
本例中为热阶段配置了 ILM 策略。 降采样发生在索引滚动并且索引时间序列结束时间（index.time_series.end_time）已过之后，因为在此之前源索引仍有望接收主要写入。 在索引结束时间过去之前，索引生命周期管理不会继续执行任何期望索引不再接收写入的操作。 在继续等待结束时间之前的索引生命周期管理操作包括： - 删除 (Delete) - 降采样 (Downsample) - 强制合并 (Force merge) - 只读 (Read only) - 可搜索快照 (Searchable snapshot) - 收缩 (Shrink)
markdown
1. PUT _ilm/policy/datastream_policy
2. {
3. "policy": {
4. "phases": {
5. "hot": {
6. "actions": {
7. "rollover" : {
8. "max_age": "5m"
9. },
10. "downsample": {
11. "fixed_interval": "1h"
12. }
13. }
14. }
15. }
16. }
17. }创建索引模板
这将为基本数据流创建索引模板。 设置时间序列数据流中详细描述了索引模板的可用参数。
为了简单起见，在时间序列映射中，所有 time_series_metric 参数都设置为 gauge 类型，但也可以使 counter 指标类型。 time_series_metric 值确定降采样期间使用的统计表示的类型。
索引模板包含一组静态时间序列维度：主机 (host)、命名空间 (namespace)、节点 (node) 和 Pod。 时间序列维度不会因降采样过程而改变。
bash
1. PUT _index_template/datastream_template
2. {
3. "index_patterns": [
4. "datastream*"
5. ],
6. "data_stream": {},
7. "template": {
8. "settings": {
9. "index": {
10. "mode": "time_series",
11. "number_of_replicas": 0,
12. "number_of_shards": 2
13. },
14. "index.lifecycle.name": "datastream_policy"
15. },
16. "mappings": {
17. "properties": {
18. "@timestamp": {
19. "type": "date"
20. },
21. "kubernetes": {
22. "properties": {
23. "container": {
24. "properties": {
25. "cpu": {
26. "properties": {
27. "usage": {
28. "properties": {
29. "core": {
30. "properties": {
31. "ns": {
32. "type": "long"
33. }
34. }
35. },
36. "limit": {
37. "properties": {
38. "pct": {
39. "type": "float"
40. }
41. }
42. },
43. "nanocores": {
44. "type": "long",
45. "time_series_metric": "gauge"
46. },
47. "node": {
48. "properties": {
49. "pct": {
50. "type": "float"
51. }
52. }
53. }
54. }
55. }
56. }
57. },
58. "memory": {
59. "properties": {
60. "available": {
61. "properties": {
62. "bytes": {
63. "type": "long",
64. "time_series_metric": "gauge"
65. }
66. }
67. },
68. "majorpagefaults": {
69. "type": "long"
70. },
71. "pagefaults": {
72. "type": "long",
73. "time_series_metric": "gauge"
74. },
75. "rss": {
76. "properties": {
77. "bytes": {
78. "type": "long",
79. "time_series_metric": "gauge"
80. }
81. }
82. },
83. "usage": {
84. "properties": {
85. "bytes": {
86. "type": "long",
87. "time_series_metric": "gauge"
88. },
89. "limit": {
90. "properties": {
91. "pct": {
92. "type": "float"
93. }
94. }
95. },
96. "node": {
97. "properties": {
98. "pct": {
99. "type": "float"
100. }
101. }
102. }
103. }
104. },
105. "workingset": {
106. "properties": {
107. "bytes": {
108. "type": "long",
109. "time_series_metric": "gauge"
110. }
111. }
112. }
113. }
114. },
115. "name": {
116. "type": "keyword"
117. },
118. "start_time": {
119. "type": "date"
120. }
121. }
122. },
123. "host": {
124. "type": "keyword",
125. "time_series_dimension": true
126. },
127. "namespace": {
128. "type": "keyword",
129. "time_series_dimension": true
130. },
131. "node": {
132. "type": "keyword",
133. "time_series_dimension": true
134. },
135. "pod": {
136. "type": "keyword",
137. "time_series_dimension": true
138. }
139. }
140. }
141. }
142. }
143. }
144. }摄取时间序列数据
使用 bulk API 请求自动创建 TSDS 并为一组 10 个文档编制索引。
重要提示：在运行此批量请求之前，你需要将时间戳更新为当前时间后三到五个小时内。 也就是说，搜索 2022-06-21T15 并替换为你当前的日期，并将小时调整为你当前的时间加三个小时。
less
1. PUT /datastream/_bulk?refresh
2. {"create": {}}
3. {"@timestamp":"2022-06-21T15:49:00Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":91153,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":463314616},"usage":{"bytes":307007078,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":585236},"rss":{"bytes":102728},"pagefaults":120901,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
4. {"create": {}}
5. {"@timestamp":"2022-06-21T15:45:50Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":124501,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":982546514},"usage":{"bytes":360035574,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1339884},"rss":{"bytes":381174},"pagefaults":178473,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
6. {"create": {}}
7. {"@timestamp":"2022-06-21T15:44:50Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":38907,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":862723768},"usage":{"bytes":379572388,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":431227},"rss":{"bytes":386580},"pagefaults":233166,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
8. {"create": {}}
9. {"@timestamp":"2022-06-21T15:44:40Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":86706,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":567160996},"usage":{"bytes":103266017,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1724908},"rss":{"bytes":105431},"pagefaults":233166,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
10. {"create": {}}
11. {"@timestamp":"2022-06-21T15:44:00Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":150069,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":639054643},"usage":{"bytes":265142477,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1786511},"rss":{"bytes":189235},"pagefaults":138172,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
12. {"create": {}}
13. {"@timestamp":"2022-06-21T15:42:40Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":82260,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":854735585},"usage":{"bytes":309798052,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":924058},"rss":{"bytes":110838},"pagefaults":259073,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
14. {"create": {}}
15. {"@timestamp":"2022-06-21T15:42:10Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":153404,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":279586406},"usage":{"bytes":214904955,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1047265},"rss":{"bytes":91914},"pagefaults":302252,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
16. {"create": {}}
17. {"@timestamp":"2022-06-21T15:40:20Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":125613,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":822782853},"usage":{"bytes":100475044,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":2109932},"rss":{"bytes":278446},"pagefaults":74843,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
18. {"create": {}}
19. {"@timestamp":"2022-06-21T15:40:10Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":100046,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":567160996},"usage":{"bytes":362826547,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1986724},"rss":{"bytes":402801},"pagefaults":296495,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
20. {"create": {}}
21. {"@timestamp":"2022-06-21T15:38:30Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":40018,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":1062428344},"usage":{"bytes":265142477,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":2294743},"rss":{"bytes":340623},"pagefaults":224530,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}针对我的情况：
less
1. PUT /datastream/_bulk?refresh
2. {"create": {}}
3. {"@timestamp":"2023-11-30T06:49:00Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":91153,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":463314616},"usage":{"bytes":307007078,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":585236},"rss":{"bytes":102728},"pagefaults":120901,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
4. {"create": {}}
5. {"@timestamp":"2023-11-30T06:45:50Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":124501,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":982546514},"usage":{"bytes":360035574,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1339884},"rss":{"bytes":381174},"pagefaults":178473,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
6. {"create": {}}
7. {"@timestamp":"2023-11-30T06:44:50Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":38907,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":862723768},"usage":{"bytes":379572388,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":431227},"rss":{"bytes":386580},"pagefaults":233166,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
8. {"create": {}}
9. {"@timestamp":"2023-11-30T06:44:40Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":86706,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":567160996},"usage":{"bytes":103266017,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1724908},"rss":{"bytes":105431},"pagefaults":233166,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
10. {"create": {}}
11. {"@timestamp":"2023-11-30T06:44:00Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":150069,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":639054643},"usage":{"bytes":265142477,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1786511},"rss":{"bytes":189235},"pagefaults":138172,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
12. {"create": {}}
13. {"@timestamp":"2023-11-30T06:42:40Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":82260,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":854735585},"usage":{"bytes":309798052,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":924058},"rss":{"bytes":110838},"pagefaults":259073,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
14. {"create": {}}
15. {"@timestamp":"2023-11-30T06:42:10Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":153404,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":279586406},"usage":{"bytes":214904955,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1047265},"rss":{"bytes":91914},"pagefaults":302252,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
16. {"create": {}}
17. {"@timestamp":"2023-11-30T06:40:20Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":125613,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":822782853},"usage":{"bytes":100475044,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":2109932},"rss":{"bytes":278446},"pagefaults":74843,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
18. {"create": {}}
19. {"@timestamp":"2023-11-30T06:40:10Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":100046,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":567160996},"usage":{"bytes":362826547,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1986724},"rss":{"bytes":402801},"pagefaults":296495,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
20. {"create": {}}
21. {"@timestamp":"2023-11-30T06:38:30Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":40018,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":1062428344},"usage":{"bytes":265142477,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":2294743},"rss":{"bytes":340623},"pagefaults":224530,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}查看结果
现在你已创建文档并将其添加到数据流中，请检查以确认新索引的当前状态。
sql
GET _data_stream如果尚未应用 ILM 策略，你的结果将如下所示。 请注意原始索引名称：.ds-datastream--000001。
json
1. {
2. "data_streams": [
3. {
4. "name": "datastream",
5. "timestamp_field": {
6. "name": "@timestamp"
7. },
8. "indices": [
9. {
10. "index_name": ".ds-datastream-2023.11.30-000001",
11. "index_uuid": "muM9y4_ORDK1wLY-t_UtnA",
12. "prefer_ilm": true,
13. "ilm_policy": "datastream_policy",
14. "managed_by": "Index Lifecycle Management"
15. }
16. ],
17. "generation": 1,
18. "status": "GREEN",
19. "template": "datastream_template",
20. "ilm_policy": "datastream_policy",
21. "next_generation_managed_by": "Index Lifecycle Management",
22. "prefer_ilm": true,
23. "hidden": false,
24. "system": false,
25. "allow_custom_routing": false,
26. "replicated": false,
27. "time_series": {
28. "temporal_ranges": [
29. {
30. "start": "2023-11-30T03:21:48.000Z",
31. "end": "2023-11-30T07:27:32.000Z"
32. }
33. ]
34. }
35. },
36. {
37. "name": "kibana_sample_data_logs",
38. "timestamp_field": {
39. "name": "@timestamp"
40. },
41. "indices": [
42. {
43. "index_name": ".ds-kibana_sample_data_logs-2023.11.21-000001",
44. "index_uuid": "JkbH5-8uTvyeZL6icuHN1A",
45. "prefer_ilm": true,
46. "managed_by": "Unmanaged"
47. }
48. ],
49. "generation": 1,
50. "status": "YELLOW",
51. "template": "kibana_sample_data_logs",
52. "next_generation_managed_by": "Unmanaged",
53. "prefer_ilm": true,
54. "hidden": false,
55. "system": false,
56. "allow_custom_routing": false,
57. "replicated": false
58. },
59. {
60. "name": "my-data-stream",
61. "timestamp_field": {
62. "name": "@timestamp"
63. },
64. "indices": [
65. {
66. "index_name": ".ds-my-data-stream-2023.11.30-000001-downsample",
67. "index_uuid": "KXtWAQCBRlW0sWYCVUu7Fw",
68. "prefer_ilm": true,
69. "managed_by": "Unmanaged"
70. },
71. {
72. "index_name": ".ds-my-data-stream-2023.11.30-000002",
73. "index_uuid": "yBDLixmpRDahmS4G5_uxQw",
74. "prefer_ilm": true,
75. "managed_by": "Unmanaged"
76. }
77. ],
78. "generation": 4,
79. "status": "GREEN",
80. "template": "my-data-stream-template",
81. "next_generation_managed_by": "Unmanaged",
82. "prefer_ilm": true,
83. "hidden": false,
84. "system": false,
85. "allow_custom_routing": false,
86. "replicated": false,
87. "time_series": {
88. "temporal_ranges": [
89. {
90. "start": "2023-11-30T02:24:20.000Z",
91. "end": "2023-11-30T08:32:32.000Z"
92. }
93. ]
94. }
95. }
96. ]
97. }接下来，运行搜索查询：
bash
GET datastream/_search
经过一段时间后，它会变成如下的结果：
json
1. {
2. "data_streams": [
3. {
4. "name": "datastream",
5. "timestamp_field": {
6. "name": "@timestamp"
7. },
8. "indices": [
9. {
10. "index_name": "downsample-1h-.ds-datastream-2023.11.30-000001",
11. "index_uuid": "KQMxcaJATf24XAesWs6Xbw",
12. "prefer_ilm": true,
13. "ilm_policy": "datastream_policy",
14. "managed_by": "Index Lifecycle Management"
15. },
16. {
17. "index_name": ".ds-datastream-2023.11.30-000002",
18. "index_uuid": "yKLevDQkQuS-8yqJQb8gWw",
19. "prefer_ilm": true,
20. "ilm_policy": "datastream_policy",
21. "managed_by": "Index Lifecycle Management"
22. }
23. ],
24. "generation": 3,
25. "status": "GREEN",
26. "template": "datastream_template",
27. "ilm_policy": "datastream_policy",
28. "next_generation_managed_by": "Index Lifecycle Management",
29. "prefer_ilm": true,
30. "hidden": false,
31. "system": false,
32. "allow_custom_routing": false,
33. "replicated": false,
34. "time_series": {
35. "temporal_ranges": [
36. {
37. "start": "2023-11-30T03:21:48.000Z",
38. "end": "2023-11-30T10:45:44.000Z"
39. }
40. ]
41. }
42. },
43. {
44. "name": "kibana_sample_data_logs",
45. "timestamp_field": {
46. "name": "@timestamp"
47. },
48. "indices": [
49. {
50. "index_name": ".ds-kibana_sample_data_logs-2023.11.21-000001",
51. "index_uuid": "JkbH5-8uTvyeZL6icuHN1A",
52. "prefer_ilm": true,
53. "managed_by": "Unmanaged"
54. }
55. ],
56. "generation": 1,
57. "status": "YELLOW",
58. "template": "kibana_sample_data_logs",
59. "next_generation_managed_by": "Unmanaged",
60. "prefer_ilm": true,
61. "hidden": false,
62. "system": false,
63. "allow_custom_routing": false,
64. "replicated": false
65. },
66. {
67. "name": "my-data-stream",
68. "timestamp_field": {
69. "name": "@timestamp"
70. },
71. "indices": [
72. {
73. "index_name": ".ds-my-data-stream-2023.11.30-000001-downsample",
74. "index_uuid": "KXtWAQCBRlW0sWYCVUu7Fw",
75. "prefer_ilm": true,
76. "managed_by": "Unmanaged"
77. },
78. {
79. "index_name": ".ds-my-data-stream-2023.11.30-000002",
80. "index_uuid": "yBDLixmpRDahmS4G5_uxQw",
81. "prefer_ilm": true,
82. "managed_by": "Unmanaged"
83. }
84. ],
85. "generation": 4,
86. "status": "GREEN",
87. "template": "my-data-stream-template",
88. "next_generation_managed_by": "Unmanaged",
89. "prefer_ilm": true,
90. "hidden": false,
91. "system": false,
92. "allow_custom_routing": false,
93. "replicated": false,
94. "time_series": {
95. "temporal_ranges": [
96. {
97. "start": "2023-11-30T02:24:20.000Z",
98. "end": "2023-11-30T10:45:44.000Z"
99. }
100. ]
101. }
102. }
103. ]
104. }
默认情况下，索引生命周期管理每十分钟检查一次是否符合策略标准。 等待大约十分钟（也许可以冲泡一杯咖啡或茶☕），然后重新运行 GET _data_stream 请求。
sql
GET _data_streamILM 策略生效后，原始 .ds-datastream-2023.11.30-000001 索引将替换为新的降采样索引，在本例中为 downsample-1h-.ds-datastream-2023.11.30-000001。
json
1. {
2. "data_streams": [
3. {
4. "name": "datastream",
5. "timestamp_field": {
6. "name": "@timestamp"
7. },
8. "indices": [
9. {
10. "index_name": "downsample-1h-.ds-datastream-2023.11.30-000001",
11. "index_uuid": "KQMxcaJATf24XAesWs6Xbw",
12. "prefer_ilm": true,
13. "ilm_policy": "datastream_policy",
14. "managed_by": "Index Lifecycle Management"
15. },
16. {
17. "index_name": ".ds-datastream-2023.11.30-000002",
18. "index_uuid": "yKLevDQkQuS-8yqJQb8gWw",
19. "prefer_ilm": true,
20. "ilm_policy": "datastream_policy",
21. "managed_by": "Index Lifecycle Management"
22. }
23. ],
24. "generation": 3,
25. "status": "GREEN",
26. "template": "datastream_template",
27. "ilm_policy": "datastream_policy",
28. "next_generation_managed_by": "Index Lifecycle Management",
29. "prefer_ilm": true,
30. "hidden": false,
31. "system": false,
32. "allow_custom_routing": false,
33. "replicated": false,
34. "time_series": {
35. "temporal_ranges": [
36. {
37. "start": "2023-11-30T03:21:48.000Z",
38. "end": "2023-11-30T10:45:44.000Z"
39. }
40. ]
41. }
42. },在数据流上运行搜索查询（请注意，在查询降采样索引时，需要注意一些细微差别）。
bash
GET datastream/_search新的降采样索引仅包含一个文档，其中包括基于原始采样指标的 min、max、sum 和 value_count 统计信息。
json
1. {
2. "took": 1,
3. "timed_out": false,
4. "_shards": {
5. "total": 4,
6. "successful": 4,
7. "skipped": 0,
8. "failed": 0
9. },
10. "hits": {
11. "total": {
12. "value": 1,
13. "relation": "eq"
14. },
15. "max_score": 1,
16. "hits": [
17. {
18. "_index": "downsample-1h-.ds-datastream-2023.11.30-000001",
19. "_id": "0eL0wC_4-45SnTNFAAABjB7QCwA",
20. "_score": 1,
21. "_source": {
22. "@timestamp": "2023-11-30T06:00:00.000Z",
23. "_doc_count": 10,
24. "kubernetes": {
25. "container": {
26. "cpu": {
27. "usage": {
28. "core": {
29. "ns": 12828317850
30. },
31. "limit": {
32. "pct": 0.0000277905
33. },
34. "nanocores": {
35. "min": 38907,
36. "max": 153404,
37. "sum": 992677,
38. "value_count": 10
39. },
40. "node": {
41. "pct": 0.0000277905
42. }
43. }
44. },
45. "memory": {
46. "available": {
47. "bytes": {
48. "min": 279586406,
49. "max": 1062428344,
50. "sum": 7101494721,
51. "value_count": 10
52. }
53. },
54. "majorpagefaults": 0,
55. "pagefaults": {
56. "min": 74843,
57. "max": 302252,
58. "sum": 2061071,
59. "value_count": 10
60. },
61. "rss": {
62. "bytes": {
63. "min": 91914,
64. "max": 402801,
65. "sum": 2389770,
66. "value_count": 10
67. }
68. },
69. "usage": {
70. "bytes": {
71. "min": 100475044,
72. "max": 379572388,
73. "sum": 2668170609,
74. "value_count": 10
75. },
76. "limit": {
77. "pct": 0.00009923134
78. },
79. "node": {
80. "pct": 0.017700378
81. }
82. },
83. "workingset": {
84. "bytes": {
85. "min": 431227,
86. "max": 2294743,
87. "sum": 14230488,
88. "value_count": 10
89. }
90. }
91. },
92. "name": "container-name-44",
93. "start_time": "2021-03-30T07:59:06.000Z"
94. },
95. "host": "gke-apps-0",
96. "namespace": "namespace26",
97. "node": "gke-apps-0-0",
98. "pod": "gke-apps-0-0-0"
99. }
100. }
101. }
102. ]
103. }
104. }使用数据流统计 API 获取数据流的统计信息，包括存储大小。
ini
GET /_data_stream/datastream/_stats?human=true
此示例演示了降采样如何作为 ILM 策略的一部分来工作，以减少指标数据的存储大小，因为它变得不那么最新且查询频率较低。
你还可以尝试我们的手动运行降采样示例，了解降采样如何在 ILM 策略之外工作。