elasticsearch学自b站的遇见狂神说,感谢老师!


安装

在阿里服务器上安装,及踩坑(服务器是1核2G)

安装

1
2
3
4
5
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.7.1-linux-x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.7.1-linux-x86_64.tar.gz.sha512
shasum -a 512 -c elasticsearch-7.7.1-linux-x86_64.tar.gz.sha512
tar -xzf elasticsearch-7.7.1-linux-x86_64.tar.gz
cd elasticsearch-7.7.1/

常见问题

异常信息(内存不足问题)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
Exception in thread "main" java.lang.RuntimeException: starting java failed with [1]
output:
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 986513408 bytes for committing reserved memory.
# An error report file with more information is saved as:
# logs/hs_err_pid24201.log
error:
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000c5330000, 986513408, 0) failed; error='Cannot allocate memory' (errno=12)
at org.elasticsearch.tools.launchers.JvmErgonomics.flagsFinal(JvmErgonomics.java:126)
at org.elasticsearch.tools.launchers.JvmErgonomics.finalJvmOptions(JvmErgonomics.java:88)
at org.elasticsearch.tools.launchers.JvmErgonomics.choose(JvmErgonomics.java:59)
at org.elasticsearch.tools.launchers.JvmOptionsParser.jvmOptions(JvmOptionsParser.java:139)
at org.elasticsearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:95)

解决

修改 config/jvm.options

1
2
-Xms128m
-Xmx128m

异常信息(不能使用root启动问题)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:174) ~[elasticsearch-7.7.1.jar:7.7.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) ~[elasticsearch-7.7.1.jar:7.7.1]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.7.1.jar:7.7.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127) ~[elasticsearch-cli-7.7.1.jar:7.7.1]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.7.1.jar:7.7.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) ~[elasticsearch-7.7.1.jar:7.7.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.7.1.jar:7.7.1]
Caused by: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:111) ~[elasticsearch-7.7.1.jar:7.7.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:178) ~[elasticsearch-7.7.1.jar:7.7.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) ~[elasticsearch-7.7.1.jar:7.7.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.7.1.jar:7.7.1]
... 6 more
uncaught exception in thread [main]
java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:111)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:178)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127)
at org.elasticsearch.cli.Command.main(Command.java:90)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)
For complete error details, refer to the log at /usr/local/elasticsearch-7.7.1/logs/elasticsearch.log
[root@iZuf65vg2nr3x1h1yzuyq1Z bin]# 2020-06-09 09:59:40,887430 UTC [25753] ERROR CNamedPipeFactory.cc@224 Unable to open named pipe /tmp/elasticsearch-48021366/controller_log_25625 for writing: Interrupted system call
2020-06-09 09:59:40,887543 UTC [25753] ERROR CLogger.cc@298 Cannot log to named pipe /tmp/elasticsearch-4802127868216853366/controller_log_25625 as it couldned for writing
2020-06-09 09:59:40,887558 UTC [25753] FATAL Main.cc@102 Could not reconfigure logging

解决

  • 新建用户组 groupadd esgroup
  • 新增用户 useradd esroot -g esgroup -p esroot
  • 设置权限 chown -R esroot:esgroup /usr/local/elasticsearch-7.7.1
  • 切换权限 su esroot
  • 运行 elasticsearch./usr/local/elasticsearch-7.7.1/bin/elasticsearch

异常信息(外网访问问题)

1
2
3
ERROR: [2] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured

解决

针对问题【1】

修改 /etc/sysctl.conf 添加

1
vm.max_map_count=655360
针对问题【2】

放开 config/elasticsearch.yml

1
cluster.initial_master_nodes: ["node-1", "node-2"]

上述异常问题:

作者: CSDN博主「小黑小黑白」

原文链接:https://blog.csdn.net/LittleBlackyoyoyo/article/details/106648664

安装head图形化界面

  1. 下载 head 插件
    注: 此插件是js开发的,需要安装 node.js

  2. 启动 head ,在根目录下执行
    这里会报错,启动不起来

    grunt server

    安装grunt

    npm install -g grunt-cli
    npm install

  3. head 连接 es
    因为js存在跨域问题,直接通过连接 localhost:9200 肯定连不上,所以需要更改 es 配置

    修改 es 的 config 目录下配置文件 config/elasticsearch.yml 增加

    1
    2
    http.cors.enabled: true
    http.cors.allow-origin: "*"

安装kibana

开箱即用

修改配置文件 ./config/kibana.yml

1
2
3
4
#加载中文语言包
i18n.locale: "zh-CN"
#因为我要本地管理homestead中的elasticsearch,所以要指定端口
elasticsearch.hosts: ["http://192.168.10.10:9200"]

在laravel中使用

引入 Composer 包

1
composer require elasticsearch/elasticsearch '~7.0'

配置

Elasticsearch 的配置很简单,我们只需要 Elasticsearch 服务器的 IP 和端口即可:

config/database.php

1
2
3
4
5
6
7
.
.
.
'elasticsearch' => [
// Elasticsearch 支持多台服务器负载均衡,因此这里是一个数组
'hosts' => explode(',', env('ES_HOSTS')),
]

我们本地环境的 Elasticsearch 的 IP 和端口是 localhost:9200,如果端口是 9200 则可以忽略不写:

.env

1
2
3
4
.
.
.
ES_HOSTS=localhost

为了保证其他的开发人员知道需要在 .env 文件中配置 ES_HOSTS,因此还需要修改一下 .env.example 文件:

.env.example

1
2
3
4
5
.
.
.

ES_HOSTS=

初始化 Elasticsearch 对象

接下来我们将初始化 Elasticsearch 对象,并注入到 Laravel 容器中:

app/Providers/AppServiceProvider.php

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
use Elasticsearch\ClientBuilder as ESClientBuilder;
.
.
.
public function register()
{
.
.
.
// 注册一个名为 es 的单例
$this->app->singleton('es', function () {
// 从配置文件读取 Elasticsearch 服务器列表
$builder = ESClientBuilder::create()->setHosts(config('database.elasticsearch.hosts'));
// 如果是开发环境
if (app()->environment() === 'local') {
// 配置日志,Elasticsearch 的请求和返回数据将打印到日志文件中,方便我们调试
$builder->setLogger(app('log')->driver());
}

return $builder->build();
});
}
.
.
.

上面为5.5以上使用,5.5配置如下

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
use Elasticsearch\ClientBuilder as ESClientBuilder;
.
.
.
public function register()
{
.
.
.
// 注册一个名为 es 的单例
$this->app->singleton('es', function () {
// 从配置文件读取 Elasticsearch 服务器列表
$builder = ESClientBuilder::create()->setHosts(config('database.elasticsearch.hosts'));
// 如果是开发环境
if (app()->environment() === 'local') {
// 配置日志,Elasticsearch 的请求和返回数据将打印到日志文件中,方便我们调试
$builder->setLogger(app('log')->getMonolog());
}

return $builder->build();
});
}
.
.
.

5.5以下的框架就自求多福吧!

测试

接下来我们来测试一下能否正常工作,首先进入 tinker:

1
php artisan tinker

然后输入:

1
>>> app('es')->info()

返回值

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
=> [
"name" => "homestead",
"cluster_name" => "homestead",
"cluster_uuid" => "zKWmBnBAQHqOiVgq_57O8w",
"version" => [
"number" => "7.7.1",
"build_flavor" => "default",
"build_type" => "deb",
"build_hash" => "ad56dce891c901a492bb1ee393f12dfff473a423",
"build_date" => "2020-05-28T16:30:01.040088Z",
"build_snapshot" => false,
"lucene_version" => "8.5.1",
"minimum_wire_compatibility_version" => "6.8.0",
"minimum_index_compatibility_version" => "6.0.0-beta1",
],
"tagline" => "You Know, for Search",
]

核心概念

  1. 索引
  2. 字段类型(mapping)
  3. 文档(documents)

elasticsearch是面向文档,关系行数据库 和 elasticsearch 客观的对比!一切都是JSON!

Relational DB Elasticsearch
数据库(database) 索引(indices)
表(tables) types
行(rows) documents
字段(columns) fields

elasticsearch(集群)中可以包含多个索引(数据库),每个索引中可以包含多个类型(表),每个类型下又包 含多 个文档(行),每个文档中又包含多个字段(列)。

Rest风格说明

一种软件架构风格,而不是标准,只是提供了一组设计原则和约束条件。它主要用于客户端和服务器交 互类的软件。基于这个风格设计的软件可以更简洁,更有层次,更易于实现缓存等机制。

method url地址 描述
PUT localhost:9200/索引名称/类型名称/文档id 创建文档(指定文档id)
POST localhost:9200/索引名称/类型名称 创建文档(随机文档id)
POST localhost:9200/索引名称/类型名称/文档id/_update 修改文档
DELETE localhost:9200/索引名称/类型名称/文档id 删除文档
GET localhost:9200/索引名称/类型名称/文档id 查询文档通过文档id
POST localhost:9200/索引名称/类型名称/_search 查询所有数据

关于索引的基本操作

创建索引

1
2
3
4
5
PUT test
{
"name": "张三",
"age": 22
}

指定字段类型

1
2
3
4
5
6
7
8
9
10
11
12
13
PUT test
{
"mappings": {
"properties": {
"name": {
"type": "text"
},
"age": {
"type": "long"
}
}
}
}

获取索引信息

1
GET test

查看默认type类型

1
2
3
4
5
PUT test/_doc/1
{
"name": "张三",
"age": 22
}
1
GET test

如果自己的文档字段没有指定,那么es 就会给我们默认配置字段类型!

修改 PUT覆盖,POST修改单个

1
2
3
4
5
PUT test/_doc/1
{
"name": "李四",
"age": 22
}
1
2
3
4
5
6
POST test/_doc/1/_update
{
"doc": {
"name": "张三"
}
}

删除索引!

1
DELETE test34

关于文档的基本操作(重点)

添加数据

1
2
3
4
5
6
7
8
9
10
11
PUT test/user/1
{
"name": "张三",
"age": 22,
"desc": "一顿操作猛如虎,一看工资2500",
"tags": [
"技术宅",
"温暖",
"直男"
]
}

获取数据 GET

1
GET test/user/1

更新数据 PUT

1
2
3
4
5
6
7
8
9
10
11
PUT test/user/1
{
"name": "李四",
"age": 25,
"desc": "一顿操作猛如虎,一看工资2500",
"tags": [
"技术宅",
"温暖",
"直男"
]
}

Post _update , 推荐使用这种更新方式!

1
2
3
4
5
6
POST test/user/1/_update
{
"doc": {
"name": "张三"
}
}

简单地搜索!

1
GET test/user/1
1
GET test/user/_search?q=name:张三

复杂操作搜索 select ( 排序,分页,高亮,模糊查询,精准查询!)

1
2
3
4
5
6
7
8
GET test/user/_search
{
"query": {
"match": {
"name": "张三"
}
}
}

输出结果,过滤字段!

1
2
3
4
5
6
7
8
9
10
11
12
GET test/user/_search
{
"query": {
"match": {
"name": "张三"
}
},
"_source": [
"name",
"age"
]
}

排序!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
GET test/user/_search
{
"query": {
"match": {
"desc": "一顿操作猛如虎,一看工资2500"
}
},
"sort": [
{
"age": {
"order": "desc"
}
}
]
}

分页查询!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
GET test/user/_search
{
"query": {
"match": {
"desc": "一顿操作猛如虎,一看工资2500"
}
},
"sort": [
{
"age": {
"order": "desc"
}
}
],
"from": 0,
"size": 20
}

数据下标还是从0开始的,和学的所有数据结构是一样的!

布尔值查询

must (and),所有的条件都要符合 where id = 1 and name = xxx

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
GET test/user/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"name": "张三"
}
},
{
"match": {
"age": 22
}
}
]
}
}
}

should(or),所有的条件都要符合 where id = 1 or name = xxx

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
GET test/user/_search
{
"query": {
"bool": {
"should": [
{
"match": {
"name": "张三"
}
},
{
"match": {
"age": 25
}
}
]
}
}
}

must_not (not)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
GET test/user/_search
{
"query": {
"bool": {
"must_not": [
{
"match": {
"age": 25
}
}
]
}
}
}

过滤器 filter(区间)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
GET test/user/_search
{
"query": {
"bool": {
"filter": [
{
"range": {
"age": {
"gte": 10,
"lte": 30
}
}
}
]
}
}
}
  • gt 大于
  • gte 大于等于
  • lt 小于
  • lte 小于等于

匹配多个条件!

多个条件使用空格隔开,只要满足其中一个结果就可以被查出

1
2
3
4
5
6
7
8
GET test/user/_search
{
"query": {
"match": {
"tags": "男 温暖"
}
}
}

精确查询!

term 查询是直接通过倒排索引指定的词条进程精确查找的!

关于分词:

  • term ,直接查询精确的
  • match,会使用分词器解析!(先分析文档,然后在通过分析的文档进行查询!)

两个类型 text keyword

  • text 会被拆分
  • keyword 字段类型不会被分词器解析

多个值匹配精确查询

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
GET test/_search
{
"query": {
"bool": {
"should": [
{
"term": {
"age":22
}
},
{
"term": {
"name": "张三"
}
}
]
}
}
}

高亮查询!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
GET test/_search
{
"query": {
"match": {
"name": "张三"
}
},
"highlight": {
"pre_tags": "<h1>",
"post_tags": "</h1>",
"fields": {
"name": {}
}
}
}

pre_tags 与 post_tags 自定义高亮html标签

这些其实MySQL 也可以做,只是MySQL 效率比较低!

  • 匹配
  • 按照条件匹配
  • 精确匹配
  • 区间范围匹配
  • 匹配字段过滤
  • 多条件查询
  • 高亮查询