Skip to content

Commit

Permalink
增加 ES 相关的笔记。
Browse files Browse the repository at this point in the history
  • Loading branch information
Suomea committed Jun 10, 2024
1 parent 54d72a2 commit f705627
Show file tree
Hide file tree
Showing 7 changed files with 277 additions and 19 deletions.
3 changes: 3 additions & 0 deletions .obsidian/page-preview.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"graph": false
}
74 changes: 55 additions & 19 deletions .obsidian/workspace.json
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,43 @@
"state": {
"type": "markdown",
"state": {
"file": "docs/Server/阿里云服务器106.15.72.83.md",
"file": "docs/Server/本地服务器192.168.31.11.md",
"mode": "source",
"source": false
}
}
},
{
"id": "58119842feb1ab9f",
"type": "leaf",
"state": {
"type": "markdown",
"state": {
"file": "docs/ES/环境搭建.md",
"mode": "source",
"source": false
}
}
},
{
"id": "33cebeca1e62f4f1",
"type": "leaf",
"state": {
"type": "markdown",
"state": {
"file": "docs/ES/Spring Boot 集成 ES.md",
"mode": "source",
"source": false
}
}
},
{
"id": "e7b94f67a9ef5845",
"type": "leaf",
"state": {
"type": "markdown",
"state": {
"file": "docs/ES/ES DSL查询方式.md",
"mode": "source",
"source": false
}
Expand All @@ -52,7 +88,7 @@
}
}
],
"currentTab": 2
"currentTab": 5
}
],
"direction": "vertical"
Expand Down Expand Up @@ -118,7 +154,7 @@
"state": {
"type": "backlink",
"state": {
"file": "docs/Server/阿里云服务器106.15.72.83.md",
"file": "docs/ES/ES DSL查询方式.md",
"collapseAll": false,
"extraContext": false,
"sortOrder": "alphabetical",
Expand All @@ -135,7 +171,7 @@
"state": {
"type": "outgoing-link",
"state": {
"file": "docs/Server/阿里云服务器106.15.72.83.md",
"file": "docs/ES/ES DSL查询方式.md",
"linksCollapsed": false,
"unlinkedCollapsed": true
}
Expand All @@ -158,7 +194,7 @@
"state": {
"type": "outline",
"state": {
"file": "docs/Server/阿里云服务器106.15.72.83.md"
"file": "docs/ES/ES DSL查询方式.md"
}
}
}
Expand All @@ -180,8 +216,21 @@
"command-palette:打开命令面板": false
}
},
"active": "3ec641745c4ff14c",
"active": "e7b94f67a9ef5845",
"lastOpenFiles": [
"docs/Docker/镜像操作.md",
"docs/Docker/安装 Docker.md",
"docs/ES/环境搭建.md",
"docs/ES/ES DSL查询方式.md",
"docs/ES/Spring Boot 集成 ES.md",
"docs/Server/本地服务器192.168.31.11.md",
"docs/ES",
"未命名.canvas",
"docs/Linux/PG Tips.md",
"docs/Linux/Nginx 编译安装.md",
"docs/Server/阿里云服务器106.15.72.83.md",
"docs/Linux/Linux Tips.md",
"docs/Linux/Debian 软件安装.md",
"docs/Java/Tips.md",
"docs/Java/微服务架构.md",
"docs/Java/微服务优雅停机.md",
Expand All @@ -197,23 +246,11 @@
"docs/Python/第 10 章:面向对象编程.md",
"docs/Python/学习资料.md",
"mkdocs.yml",
"docs/Server/阿里云服务器106.15.72.83.md",
"docs/Linux/正则表达式.md",
"docs/Kafka/Kafka 常用命令.md",
"docs/Network/运输层:UDP.md",
"docs/Server",
"docs/OS/GDB.md",
"docs/Linux/Linux Tips.md",
"docs/Kafka/Kafka 集群搭建.md",
"docs/Kafka/Kafka 主题与分区.md",
"docs/Network/运输层:TCP.md",
"docs/Network/网络层:IP.md",
"docs/Network/Network Tips.md",
"docs/Linux/网络.md",
"docs/OS/C.md~",
"docs/LocalFile/Picture/Pastedimage20240423194526.png",
"docs/LocalFile/Picture/Pastedimage20240423194147.png",
"未命名.canvas",
"docs/LocalFile/Picture/TCP接收窗口和分配的缓存.png",
"docs/LocalFile/Picture/TCP发送窗口.png",
"site/Tips/API/index.html",
Expand All @@ -222,7 +259,6 @@
"site/Tips/深度学习/index.html",
"site/Tips/开发规范/index.html",
"site/Tips/深度学习",
"site/Tips/开发规范",
"site/assets/images/favicon.png",
"site/LocalFile/Picture/选项的格式.png",
"site/LocalFile/Picture/详细分分片示例.png",
Expand Down
15 changes: 15 additions & 0 deletions docs/Docker/安装 Docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,18 @@ curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh
```

## 网络模式
Docker 安装时默认会创建三个网络,同时在宿主机上创建一个虚拟网卡 docker0。
```shell
# docker network list
NETWORK ID NAME DRIVER SCOPE
b092ef563276 bridge bridge local
e8eb78a7d5ff host host local
e1290a9d96db none null local
```

host 使用是宿主机的网络。

bridge 容器使用独立的 network namespace,并连接到 docker0 虚拟网卡。默认。

none 容器有独立的 network namespace,但是不进行任何网络设置。
88 changes: 88 additions & 0 deletions docs/ES/ES DSL查询方式.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
查询所有:查询出所有数据,一般测试使用,match_all,不会返回全部数据,有限制条数啦。
全文检索:利用分词器对查询文本进行分词,然后匹配倒排索引库,然后拿到文档的 ID,计算得分,根据 ID 查询文档。
- match_query
- match
- multi_match_query
精确查询:根据精确的词条查找数据,一般查询 keyword、数值、日期、boolean 等类型字段。
- ids
- range
- term
地理查询:根据经纬度查询。
- geo_distance
- geo_bounding_box
复合查询:将上述的查询条件组合起来,合并查询条件。
- bool
- function_score

基本语法
```
GET /indexName/_search
{
"query": {
"查询类型": {
"查询条件": "条件值"
}
}
}
```

查询示例
```
# 查询全部
GET /book/_search
{
"query": {
"match_all": {}
}
}
# 全文检索,单字段
GET /book/_search
{
"query": {
"match": {
"all": "Java"
}
}
}
# 全文检索,多字段,可以使用 * 表示全部字段
GET /book/_search
{
"query": {
"multi_match": {
"query": "Java",
"fields": ["name", "author"]
}
}
}
# 精确查询词条查询
GET /book/_search
{
"query": {
"term": {
"name": {
"value": "Java 揭秘"
}
}
}
}
# 精确查询范围查询
GET /boot/_search
{
"query": {
"range": {
"public_year": {
"gte": 2013,
"lte": 2034
}
}
}
}
```
Empty file.
98 changes: 98 additions & 0 deletions docs/ES/环境搭建.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
创建网络
```shell
docker network create elastic-net
```

配置密码
```
export ELASTIC_PASSWORD="<ES_PASSWORD>" # password for "elastic" username
export KIBANA_PASSWORD="<KIB_PASSWORD>" # Used _internally_ by Kibana, must be at least 6 characters long
```

Docker 安装 ES
```shell
docker run -p 9200:9200 -itd --name elasticsearch --network elastic-net \
-e ELASTIC_PASSWORD=$ELASTIC_PASSWORD \
-e "discovery.type=single-node" \
-e "xpack.security.http.ssl.enabled=false" \
-e "xpack.license.self_generated.type=basic" \
docker.elastic.co/elasticsearch/elasticsearch:8.12.2
```



Docker 安装 Kibana

初始化 `kibana_system` 用户密码
```shell
curl -u elastic:$ELASTIC_PASSWORD \
-X POST \
http://localhost:9200/_security/user/kibana_system/_password \
-d '{"password":"'"$KIBANA_PASSWORD"'"}' \
-H 'Content-Type: application/json'
```

安装 Kibana
```shell
docker run -p 5601:5601 -itd --name kibana --network elastic-net \
-e ELASTICSEARCH_URL=http://elasticsearch:9200 \
-e ELASTICSEARCH_HOSTS=http://elasticsearch:9200 \
-e ELASTICSEARCH_USERNAME=kibana_system \
-e ELASTICSEARCH_PASSWORD=$KIBANA_PASSWORD \
-e "xpack.security.enabled=false" \
-e "xpack.license.self_generated.type=basic" \
docker.elastic.co/kibana/kibana:8.12.2
```

完成安装了 ES 和 Kibana,
ES 地址:http://192.168.31.11:9200,
ES 有两个用户
- 管理员用户 elastic/$ELASTIC_PASSWORD。
- Kibana 容器连接用户 kibana_system/$KIBANA_PASSWORD
Kibana 地址:http://192.168.31.11:5601,使用 ES 管理员用户登录即可。

ES 配置中文分词器
下载分词器安装包,https://github.com/infinilabs/analysis-ik/releases,注意版本匹配。
```shell
unzip elasticsearch-analysis-ik-8.12.2.zip -d ik
docker cp ik/ elasticsearch:/usr/share/elasticsearch/plugins/
docker restart elasticsearch
```

测试 ik 分词器
```text
POST /_analyze
{
"analyzer": "standard",
"text": "nice to meet you 我是程序员!"
}
POST /_analyze
{
"analyzer": "ik_smart",
"text": "nice to meet you 我是程序员!"
}
POST /_analyze
{
"analyzer": "ik_max_word",
"text": "nice to meet you 我是程序员!"
}
```

ik 分词器支持配置扩展词和停止词。如果 ik 自带的字典里面没有相关词汇,默认分成一个一个字,扩展词典可以配置相关的词避免分成一个一个字。停止词可以配置一些语气词或者冠词或者一些敏感词,那么分词的时候这些词就会忽略掉。
```xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
<comment>IK Analyzer 扩展配置</comment>
<!--用户可以在这里配置自己的扩展字典 -->
<entry key="ext_dict"></entry>
<!--用户可以在这里配置自己的扩展停止词字典-->
<entry key="ext_stopwords"></entry>
<!--用户可以在这里配置远程扩展字典 -->
<!-- <entry key="remote_ext_dict">words_location</entry> -->
<!--用户可以在这里配置远程扩展停止词字典-->
<!-- <entry key="remote_ext_stopwords">words_location</entry> -->
</properties>
```
18 changes: 18 additions & 0 deletions docs/Server/本地服务器192.168.31.11.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
本地测试服务器,4核心 8 线程,16GB 内存,50GB 磁盘存储。Debian GNU/Linux 12 (bookworm) 系统。

配置了 APT 清华源。
安装了 VIM。
配置了 root 允许密码登录。
安装了 Docker,参考 [[安装 Docker]]
配置静态IP。

一、ElasticSearch 安装。

二、Kbina 安装。



| 端口 | 服务 | 描述 |
| ---- | ----------------- | ---------- |
| 9200 | ElasticSearch | Docker 安装。 |
| 2000 | Java web-tools 服务 | 提供开发工具接口 |

0 comments on commit f705627

Please sign in to comment.