diff --git a/docs/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md b/docs/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md index c3fd0691ea599..0f03ed0134374 100644 --- a/docs/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md +++ b/docs/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md @@ -27,127 +27,159 @@ under the License. ## Description -This statement creates a SQL blocking rule. it can restrict any kind of sql statements(no matter DDL or DML statement). +This statement is used to create an SQL block rule. -Supports configuring SQL blacklists by user: +SQL_BLOCK_RULE can be used to restrict users from performing certain operations, such as avoiding scanning large amounts of data. -- Refuse to specify SQL by regular matching -- Check if a sql reaches one of these limits by setting partition_num, tablet_num, cardinality - - partition_num, tablet_num, cardinality can be set together, once a query reaches one of these limits, the query will be intercepted - -grammar: +## Syntax ```sql -CREATE SQL_BLOCK_RULE rule_name -[PROPERTIES ("key"="value", ...)]; +CREATE SQL_BLOCK_RULE +PROPERTIES ( + -- property + + -- Additional properties + [ , ... ] + ) ``` -Parameter Description: +## Required Parameters + +**1. ``** + +> The name of the rule. + +**2. ``** + +> The properties of the rule can be divided into three categories: SQL execution, scan limitation, and switch. + +> The SQL execution and scan limitation categories are mutually exclusive, meaning that an SQL block rule can only restrict one of them. + +> **SQL Execution Category** + +> There are two types, representing regular expression matching and exact matching, respectively. Only one of them can be chosen. +> +> - sql: The matching rule (based on regular expression matching, special characters need to be escaped, for example, `select *` should be written as `select \\*`). When a user executes an SQL statement, the system will use the SQL set here as a regular expression to match the SQL submitted by the user. If it matches, the execution of the SQL will be blocked. +> - sqlHash: The MD5 hash value of the SQL. This is mainly used in conjunction with slow logs. Users do not need to calculate the hash value themselves. For example, if a slow log shows that a particular SQL is running slowly, you can copy the `SqlHash` from `fe.audit.log` and create an SQL_BLOCK_RULE to restrict the execution of this SQL. +> +> **Scan Limitation Category** +> When a user initiates a query, the query optimizer will calculate the number of partitions, tablets, and rows of data that need to be scanned for each table. The following properties can be used to limit these three numbers, either all at once or just some of them. +> - partition_num: The maximum number of partitions that a table will scan. +> - tablet_num: The maximum number of tablets that a table will scan. +> - cardinality: The number of rows of data that a table will scan. +> +> **Switch Category** +> +> - global: Whether the rule is effective for all users. The default is false. If it is not set to true, the rule needs to be applied to a specific user through the `set property` command. +> - enable: Whether the blocking rule is enabled. The default is true. -- sql: matching rule (based on regular matching, special characters need to be translated,for example`select *`use`select \\*`), optional, the default value is "NULL" -- sqlHash: sql hash value, used for exact matching, we will print this value in `fe.audit.log`, optional, this parameter and sql can only be selected one, the default value is "NULL" -- partition_num: the maximum number of partitions a scan node will scan, the default value is 0L -- tablet_num: The maximum number of tablets that a scanning node will scan, the default value is 0L -- cardinality: the rough scan line number of a scan node, the default value is 0L -- global: Whether to take effect globally (all users), the default is false -- enable: whether to enable blocking rules, the default is true + +## Access Control Requirements + +The user executing this SQL command must have at least the following permissions: + +| Privilege | Object | Notes | +| ------------ | ------ | ----- | +| ADMIN_PRIV | Global | | ## Example -1. Create a block rule named test_rule +1. Create a rule that prevents all users from executing `select * from order_analysis` - ```sql - CREATE SQL_BLOCK_RULE test_rule - PROPERTIES( - "sql"="select \\* from order_analysis", - "global"="false", - "enable"="true" - ); - ``` + ```sql + CREATE SQL_BLOCK_RULE test_rule + PROPERTIES( + "sql"="select \\* from order_analysis", + "global"="true", + "enable"="true" + ); + ``` + + When we execute the SQL defined in the rule, it will return an exception error, as shown below: - >Notes: - > - >That the sql statement here does not end with a semicolon - - When we execute the sql we just defined in the rule, an exception error will be returned. The example is as follows: - - ```sql - select * from order_analysis; - ERROR 1064 (HY000): errCode = 2, detailMessage = sql match regex sql block rule: order_analysis_rule - ``` + ```sql + mysql> select * from order_analysis; + ERROR 1064 (HY000): errCode = 2, detailMessage = sql match regex sql block rule: order_analysis_rule + ``` + +2. Create a rule that prevents scanning more than 30 partitions of the same table and restricts the query data volume to no more than 10 billion rows -2. Create test_rule2, limit the maximum number of scanned partitions to 30, and limit the maximum scan base to 10 billion rows. The example is as follows: ```sql - CREATE SQL_BLOCK_RULE test_rule2 - PROPERTIES ( - "partition_num" = "30", - "cardinality" = "10000000000", - "global" = "false", - "enable" = "true" - ); + CREATE SQL_BLOCK_RULE test_rule2 + PROPERTIES + ( + "partition_num" = "30", + "cardinality" = "10000000000", + "global" = "true", + "enable" = "true" + ); ``` -3. Create SQL BLOCK RULE with special chars - - ```sql - CREATE SQL_BLOCK_RULE test_rule3 - PROPERTIES - ( - "sql" = "select count\\(1\\) from db1.tbl1" - ); - CREATE SQL_BLOCK_RULE test_rule4 - PROPERTIES - ( - "sql" = "select \\* from db1.tbl1" - ); - ``` -4. In SQL_BLOCK_RULE, SQL matching is based on regular expressions. If want to match more patterns of SQL, need to write the corresponding regex. For example, to ignore spaces in SQL and not query tables that start with 'order_', as shown below: - - ```sql - CREATE SQL_BLOCK_RULE test_rule4 - PROPERTIES( - "sql"="\\s*select\\s*\\*\\s*from order_\\w*\\s*", + +3. SQL matching is based on regular expressions. If you want to match more patterns of SQL, you need to write the corresponding regular expressions. For example, ignoring spaces in SQL and preventing queries on tables starting with "order", as shown below: + + + ```sql + CREATE SQL_BLOCK_RULE test_rule3 + PROPERTIES( + "sql"="\\s*select\\s*\\*\\s*from order_\\w*\\s*", + "global"="true", + "enable"="true" + ); + ``` + +4. Create a rule that is only applicable to specific users + + + ```sql + CREATE SQL_BLOCK_RULE test_rule4 + PROPERTIES( + "sql"="select \\* from order_analysis", "global"="false", "enable"="true" - ); - ``` + ); + ``` + + Apply the rule to user jack -### APPENDIX -Here are some commonly used regular expressions: -> . :Matches any single character except for a newline character \n. -> -> * :Matches the preceding element zero or more times. For example, a matches zero or more 'a'. -> -> + :Matches the preceding element one or more times. For example, a+ matches one or more 'a'. -> -> ? :Matches the preceding element zero or one time. For example, a? matches zero or one 'a'. -> -> [] :Used to define a character set. For example, [aeiou] matches any one vowel letter. -> -> [^] :In a character set, use ^ to indicate negation, matching characters that are not in the set. For example, [^0-9] matches any non-digit character. -> -> () :Used for grouping expressions and applying quantifiers. For example, (ab)+ matches consecutive 'ab'. -> -> | :Represents logical OR. For example, a|b matches 'a' or 'b'. -> -> ^ :Matches the beginning of a string. For example, ^abc matches a string that starts with 'abc'. -> -> $ :Matches the end of a string. For example, xyz$ matches a string that ends with 'xyz'. -> -> \ :Used to escape special characters to treat them as ordinary characters. For example, \\. matches the period character '.'. -> -> \s :Matches any whitespace character, including spaces, tabs, newline characters, etc. -> -> \d :Matches any digit character, equivalent to [0-9]. -> -> \w :Matches any word character, including letters, digits, and underscores, equivalent to [a-zA-Z0-9_]. -## Keywords + ```sql + SET PROPERTY FOR 'jack' 'sql_block_rules' = 'test_rule4'; + ``` + +## Others + +Common regular expressions are as follows: + +TextCopy ```text -CREATE, SQL_BLCOK_RULE -``` +. : Matches any single character except the newline character `\n`. + +* : Matches the preceding element zero or more times. For example, a* matches zero or more 'a's. + ++ : Matches the preceding element one or more times. For example, a+ matches one or more 'a's. + +? : Matches the preceding element zero or one time. For example, a? matches zero or one 'a'. + +[] : Defines a character set. For example, [aeiou] matches any vowel. + +[^] : When used in a character set, ^ indicates negation, matching characters not in the set. For example, [^0-9] matches any non-digit character. + +() : Groups expressions, allowing quantifiers to be applied to them. For example, (ab)+ matches consecutive 'ab's. + +| : Indicates logical OR. For example, a|b matches 'a' or 'b'. + +^ : Matches the beginning of a string. For example, ^abc matches strings starting with 'abc'. + +$ : Matches the end of a string. For example, xyz$ matches strings ending with 'xyz'. + +\ : Escapes special characters, making them ordinary characters. For example, \\. matches the period character '.'. + +\s : Matches any whitespace character, including spaces, tabs, newlines, etc. -## Best Practice +\d : Matches any digit character, equivalent to [0-9]. +\w : Matches any word character, including letters, digits, and underscores, equivalent to [a-zA-Z0-9_]. +``` \ No newline at end of file diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md index bc57f8d567acd..a78cb38dafda0 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md @@ -26,131 +26,155 @@ under the License. - - ## 描述 -该语句创建 SQL 阻止规则,该功能可用于限制任何 sql 语句(包括 DDL 和 DML 语句)。 - -支持按用户配置 SQL 黑名单: +该语句用于创建 SQL 阻止规则 -- 通过正则匹配的方式拒绝指定 SQL -- 通过设置 partition_num, tablet_num, cardinality, 检查一个查询是否达到其中一个限制 - - partition_num, tablet_num, cardinality 可以一起设置,一旦一个查询达到其中一个限制,查询将会被拦截 +SQL_BLOCK_RULE 可用于限制用户执行某些操作,比如避免扫描大量数据等 -语法: +## 语法 ```sql -CREATE SQL_BLOCK_RULE rule_name -[PROPERTIES ("key"="value", ...)]; +CREATE SQL_BLOCK_RULE +PROPERTIES ( + -- property + + -- Additional properties + [ , ... ] + ) ``` -参数说明: +## 必选参数 + +**1. ``** + +> 规则的名字 + +**2. ``** + +> 规则的属性,可以分为三类:执行 SQL 类,扫描限制类和开关类。 +> +> 执行 SQL 类和扫描限制类是互斥的,也就是说一个 SQL_BLOCK_RULE 只能限制其中一种。 +> +> **执行 SQL 类:** +> +>包含两种,分别代表正则匹配和精准匹配,两种只能选择其中一个。 +> +> - sql:匹配规则 (基于正则匹配,特殊字符需要转译,如`select *`使用`select \\*`),用户在执行 sql 时,系统会把这里设置的 sql 作为正则对用户提交的 sql 进行匹配,如果能匹配上,会阻止 sql 的运行。 +> +> - sqlHash: sql 的 md5 摘要值,这里主要是配合慢日志来使用,用户无需自己计算摘要值,例如慢日志中发现某个 sql 执行的很慢,可以在`fe.audit.log`复制`SqlHash`,创建 SQL_BLOCK_RULE 限制这个 sql 运行。 +> +> **扫描限制类** +> +> 用户发起查询的时候,查询优化器会计算出每张表需要扫描的 partition 数,tablet 数 和扫描的数据行数。以下属性分别对这三个数字进行限制,可以同时设置,也可以只限制部分 +> +> - partition_num: 一个表将扫描的最大 partition 数量 +> - tablet_num: 一个表将扫描的最大 tablet 数量 +> - cardinality: 一个表将扫描的数据行数 +> +> **开关类** +> +> - global:是否全局 (所有用户) 生效,默认为 false。不设置为 true 的话,需要通过 set property 对某个用户生效 +> - enable:是否开启阻止规则,默认为 true -- sql:匹配规则 (基于正则匹配,特殊字符需要转译,如`select *`使用`select \\*`),可选,默认值为 "NULL", 最后不要带分号 -- sqlHash: sql hash 值,用于完全匹配,我们会在`fe.audit.log`打印这个值,可选,这个参数和 sql 只能二选一,默认值为 "NULL" -- partition_num: 一个扫描节点会扫描的最大 partition 数量,默认值为 0L -- tablet_num: 一个扫描节点会扫描的最大 tablet 数量,默认值为 0L -- cardinality: 一个扫描节点粗略的扫描行数,默认值为 0L -- global:是否全局 (所有用户) 生效,默认为 false -- enable:是否开启阻止规则,默认为 true +## 权限控制 + +执行此 SQL 命令的用户必须至少具有以下权限: + +| 权限 | 对象 | 说明 | +| :---------------- | :------------- | :------------ | +| ADMIN_PRIV | 全局 | | ## 示例 -1. 创建一个名称为 test_rule 的阻止规则 +1. 创建不允许所有用户执行 `select * from order_analysis`的规则 + + ```sql + CREATE SQL_BLOCK_RULE test_rule + PROPERTIES( + "sql"="select \\* from order_analysis", + "global"="true", + "enable"="true" + ); + ``` - ```sql - CREATE SQL_BLOCK_RULE test_rule - PROPERTIES( - "sql"="select \\* from order_analysis", - "global"="false", - "enable"="true" - ); - ``` - 当我们去执行刚才我们定义在规则里的 sql 时就会返回异常错误,示例如下: + 当我们去执行刚才定义在规则里的 sql 时就会返回异常错误,示例如下: - ```sql - mysql> select * from order_analysis; - ERROR 1064 (HY000): errCode = 2, detailMessage = sql match regex sql block rule: order_analysis_rule - ``` + ```sql + mysql> select * from order_analysis; + ERROR 1064 (HY000): errCode = 2, detailMessage = sql match regex sql block rule: order_analysis_rule + ``` -2. 创建 test_rule2,将最大扫描的分区数量限制在 30 个,最大扫描基数限制在 100 亿行,示例如下: +2. 创建不允许对同一个表扫描的分区数量超过 30 并且查询的数据量不能超过 100 亿行 ```sql CREATE SQL_BLOCK_RULE test_rule2 PROPERTIES ( - "partition_num" = "30", - "cardinality" = "10000000000", - "global" = "false", - "enable" = "true" + "partition_num" = "30", + "cardinality" = "10000000000", + "global" = "true", + "enable" = "true" ); - Query OK, 0 rows affected (0.01 sec) - ``` + ``` +3. SQL 的匹配是基于正则的,如果想匹配更多模式的 SQL 需要写相应的正则,比如忽略 SQL 中空格,还有 order 开头的表都不能查询,示例如下: -3. 创建包含特殊字符的 SQL BLOCK RULE,正则表达式中 ( 和 ) 符号是特殊符号,所以需要转义,示例如下: - ```sql CREATE SQL_BLOCK_RULE test_rule3 - PROPERTIES - ( - "sql" = "select count\\(1\\) from db1.tbl1" - ); - CREATE SQL_BLOCK_RULE test_rule4 - PROPERTIES - ( - "sql" = "select \\* from db1.tbl1" + PROPERTIES( + "sql"="\\s*select\\s*\\*\\s*from order_\\w*\\s*", + "global"="true", + "enable"="true" ); ``` - -4. SQL_BLCOK_RULE 中,SQL 的匹配是基于正则的,如果想匹配更多模式的 SQL 需要写相应的正则,比如忽略 SQL -中空格,还有 order_ 开头的表都不能查询,示例如下: - - ```sql - CREATE SQL_BLOCK_RULE test_rule4 - PROPERTIES( - "sql"="\\s*select\\s*\\*\\s*from order_\\w*\\s*", - "global"="false", - "enable"="true" - ); - ``` - -### 附录 -常用正则表达式如下: -> . :匹配任何单个字符,除了换行符 `\n`。 -> -> * :匹配前面的元素零次或多次。例如,a* 匹配零个或多个 'a'。 -> -> + :匹配前面的元素一次或多次。例如,a+ 匹配一个或多个 'a'。 -> -> ? :匹配前面的元素零次或一次。例如,a? 匹配零个或一个 'a'。 -> -> [] :用于定义字符集合。例如,[aeiou] 匹配任何一个元音字母。 -> -> [^] :在字符集合中使用 ^ 表示否定,匹配不在集合内的字符。例如,[^0-9] 匹配任何非数字字符。 -> -> () :用于分组表达式,可以对其应用量词。例如,(ab)+ 匹配连续的 'ab'。 -> -> | :用于表示或逻辑。例如,a|b 匹配 'a' 或 'b'。 -> -> ^ :匹配字符串的开头。例如,^abc 匹配以 'abc' 开头的字符串。 -> -> $ :匹配字符串的结尾。例如,xyz$ 匹配以 'xyz' 结尾的字符串。 -> -> \ :用于转义特殊字符,使其变成普通字符。例如,\\. 匹配句点字符 '.'。 -> -> \s:匹配任何空白字符,包括空格、制表符、换行符等。 -> -> \d:匹配任何数字字符,相当于 [0-9]。 -> -> \w:匹配任何单词字符,包括字母、数字和下划线,相当于 [a-zA-Z0-9_]。 +4. 创建只针对部分用户的规则 + + ```sql + CREATE SQL_BLOCK_RULE test_rule4 + PROPERTIES( + "sql"="select \\* from order_analysis", + "global"="false", + "enable"="true" + ); + ``` + + 将规则作用于用户 jack + + ```sql + SET PROPERTY FOR 'jack' 'sql_block_rules' = 'test_rule4'; + ``` -## 关键词 +## 其它 + +常用正则表达式如下: ```text -CREATE, SQL_BLCOK_RULE -``` +. :匹配任何单个字符,除了换行符 `\n`。 + +* :匹配前面的元素零次或多次。例如,a* 匹配零个或多个 'a'。 + ++ :匹配前面的元素一次或多次。例如,a+ 匹配一个或多个 'a'。 + +? :匹配前面的元素零次或一次。例如,a? 匹配零个或一个 'a'。 + +[] :用于定义字符集合。例如,[aeiou] 匹配任何一个元音字母。 + +[^] :在字符集合中使用 ^ 表示否定,匹配不在集合内的字符。例如,[^0-9] 匹配任何非数字字符。 + +() :用于分组表达式,可以对其应用量词。例如,(ab)+ 匹配连续的 'ab'。 + +| :用于表示或逻辑。例如,a|b 匹配 'a' 或 'b'。 + +^ :匹配字符串的开头。例如,^abc 匹配以 'abc' 开头的字符串。 + +$ :匹配字符串的结尾。例如,xyz$ 匹配以 'xyz' 结尾的字符串。 + +\ :用于转义特殊字符,使其变成普通字符。例如,\\. 匹配句点字符 '.'。 + +\s:匹配任何空白字符,包括空格、制表符、换行符等。 -### 最佳实践 +\d:匹配任何数字字符,相当于 [0-9]。 +\w:匹配任何单词字符,包括字母、数字和下划线,相当于 [a-zA-Z0-9_]。 +``` \ No newline at end of file diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md index 34138b2bf21be..5a736d55cb0eb 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md @@ -26,129 +26,156 @@ under the License. -## 描述 -该语句创建 SQL 阻止规则,该功能可用于限制任何 sql 语句(包括 DDL 和 DML 语句)。 +## 描述 -支持按用户配置 SQL 黑名单: +该语句用于创建 SQL 阻止规则 -- 通过正则匹配的方式拒绝指定 SQL -- 通过设置 partition_num, tablet_num, cardinality, 检查一个查询是否达到其中一个限制 - - partition_num, tablet_num, cardinality 可以一起设置,一旦一个查询达到其中一个限制,查询将会被拦截 +SQL_BLOCK_RULE 可用于限制用户执行某些操作,比如避免扫描大量数据等 -语法: +## 语法 ```sql -CREATE SQL_BLOCK_RULE rule_name -[PROPERTIES ("key"="value", ...)]; +CREATE SQL_BLOCK_RULE +PROPERTIES ( + -- property + + -- Additional properties + [ , ... ] + ) ``` -参数说明: +## 必选参数 + +**1. ``** + +> 规则的名字 + +**2. ``** + +> 规则的属性,可以分为三类:执行 SQL 类,扫描限制类和开关类。 +> +> 执行 SQL 类和扫描限制类是互斥的,也就是说一个 SQL_BLOCK_RULE 只能限制其中一种。 +> +> **执行 SQL 类:** +> +>包含两种,分别代表正则匹配和精准匹配,两种只能选择其中一个。 +> +> - sql:匹配规则 (基于正则匹配,特殊字符需要转译,如`select *`使用`select \\*`),用户在执行 sql 时,系统会把这里设置的 sql 作为正则对用户提交的 sql 进行匹配,如果能匹配上,会阻止 sql 的运行。 +> +> - sqlHash: sql 的 md5 摘要值,这里主要是配合慢日志来使用,用户无需自己计算摘要值,例如慢日志中发现某个 sql 执行的很慢,可以在`fe.audit.log`复制`SqlHash`,创建 SQL_BLOCK_RULE 限制这个 sql 运行。 +> +> **扫描限制类** +> +> 用户发起查询的时候,查询优化器会计算出每张表需要扫描的 partition 数,tablet 数 和扫描的数据行数。以下属性分别对这三个数字进行限制,可以同时设置,也可以只限制部分 +> +> - partition_num: 一个表将扫描的最大 partition 数量 +> - tablet_num: 一个表将扫描的最大 tablet 数量 +> - cardinality: 一个表将扫描的数据行数 +> +> **开关类** +> +> - global:是否全局 (所有用户) 生效,默认为 false。不设置为 true 的话,需要通过 set property 对某个用户生效 +> - enable:是否开启阻止规则,默认为 true + +## 权限控制 + +执行此 SQL 命令的用户必须至少具有以下权限: -- sql:匹配规则 (基于正则匹配,特殊字符需要转译,如`select *`使用`select \\*`),可选,默认值为 "NULL", 最后不要带分号 -- sqlHash: sql hash 值,用于完全匹配,我们会在`fe.audit.log`打印这个值,可选,这个参数和 sql 只能二选一,默认值为 "NULL" -- partition_num: 一个扫描节点会扫描的最大 partition 数量,默认值为 0L -- tablet_num: 一个扫描节点会扫描的最大 tablet 数量,默认值为 0L -- cardinality: 一个扫描节点粗略的扫描行数,默认值为 0L -- global:是否全局 (所有用户) 生效,默认为 false -- enable:是否开启阻止规则,默认为 true +| 权限 | 对象 | 说明 | +| :---------------- | :------------- | :------------ | +| ADMIN_PRIV | 全局 | | ## 示例 -1. 创建一个名称为 test_rule 的阻止规则 +1. 创建不允许所有用户执行 `select * from order_analysis`的规则 + + ```sql + CREATE SQL_BLOCK_RULE test_rule + PROPERTIES( + "sql"="select \\* from order_analysis", + "global"="true", + "enable"="true" + ); + ``` - ```sql - CREATE SQL_BLOCK_RULE test_rule - PROPERTIES( - "sql"="select \\* from order_analysis", - "global"="false", - "enable"="true" - ); - ``` - 当我们去执行刚才我们定义在规则里的 sql 时就会返回异常错误,示例如下: + 当我们去执行刚才定义在规则里的 sql 时就会返回异常错误,示例如下: - ```sql - mysql> select * from order_analysis; - ERROR 1064 (HY000): errCode = 2, detailMessage = sql match regex sql block rule: order_analysis_rule - ``` + ```sql + mysql> select * from order_analysis; + ERROR 1064 (HY000): errCode = 2, detailMessage = sql match regex sql block rule: order_analysis_rule + ``` -2. 创建 test_rule2,将最大扫描的分区数量限制在 30 个,最大扫描基数限制在 100 亿行,示例如下: +2. 创建不允许对同一个表扫描的分区数量超过 30 并且查询的数据量不能超过 100 亿行 ```sql CREATE SQL_BLOCK_RULE test_rule2 PROPERTIES ( - "partition_num" = "30", - "cardinality" = "10000000000", - "global" = "false", - "enable" = "true" + "partition_num" = "30", + "cardinality" = "10000000000", + "global" = "true", + "enable" = "true" ); - Query OK, 0 rows affected (0.01 sec) - ``` + ``` +3. SQL 的匹配是基于正则的,如果想匹配更多模式的 SQL 需要写相应的正则,比如忽略 SQL 中空格,还有 order 开头的表都不能查询,示例如下: -3. 创建包含特殊字符的 SQL BLOCK RULE,正则表达式中 ( 和 ) 符号是特殊符号,所以需要转义,示例如下: - ```sql CREATE SQL_BLOCK_RULE test_rule3 - PROPERTIES - ( - "sql" = "select count\\(1\\) from db1.tbl1" - ); - CREATE SQL_BLOCK_RULE test_rule4 - PROPERTIES - ( - "sql" = "select \\* from db1.tbl1" + PROPERTIES( + "sql"="\\s*select\\s*\\*\\s*from order_\\w*\\s*", + "global"="true", + "enable"="true" ); ``` - -4. SQL_BLCOK_RULE 中,SQL 的匹配是基于正则的,如果想匹配更多模式的 SQL 需要写相应的正则,比如忽略 SQL -中空格,还有 order_ 开头的表都不能查询,示例如下: - - ```sql - CREATE SQL_BLOCK_RULE test_rule4 - PROPERTIES( - "sql"="\\s*select\\s*\\*\\s*from order_\\w*\\s*", - "global"="false", - "enable"="true" - ); - ``` - -### 附录 -常用正则表达式如下: -> . :匹配任何单个字符,除了换行符 `\n`。 -> -> * :匹配前面的元素零次或多次。例如,a* 匹配零个或多个 'a'。 -> -> + :匹配前面的元素一次或多次。例如,a+ 匹配一个或多个 'a'。 -> -> ? :匹配前面的元素零次或一次。例如,a? 匹配零个或一个 'a'。 -> -> [] :用于定义字符集合。例如,[aeiou] 匹配任何一个元音字母。 -> -> [^] :在字符集合中使用 ^ 表示否定,匹配不在集合内的字符。例如,[^0-9] 匹配任何非数字字符。 -> -> () :用于分组表达式,可以对其应用量词。例如,(ab)+ 匹配连续的 'ab'。 -> -> | :用于表示或逻辑。例如,a|b 匹配 'a' 或 'b'。 -> -> ^ :匹配字符串的开头。例如,^abc 匹配以 'abc' 开头的字符串。 -> -> $ :匹配字符串的结尾。例如,xyz$ 匹配以 'xyz' 结尾的字符串。 -> -> \ :用于转义特殊字符,使其变成普通字符。例如,\\. 匹配句点字符 '.'。 -> -> \s:匹配任何空白字符,包括空格、制表符、换行符等。 -> -> \d:匹配任何数字字符,相当于 [0-9]。 -> -> \w:匹配任何单词字符,包括字母、数字和下划线,相当于 [a-zA-Z0-9_]。 +4. 创建只针对部分用户的规则 + + ```sql + CREATE SQL_BLOCK_RULE test_rule4 + PROPERTIES( + "sql"="select \\* from order_analysis", + "global"="false", + "enable"="true" + ); + ``` + + 将规则作用于用户 jack + + ```sql + SET PROPERTY FOR 'jack' 'sql_block_rules' = 'test_rule4'; + ``` -## 关键词 +## 其它 + +常用正则表达式如下: ```text -CREATE, SQL_BLCOK_RULE -``` +. :匹配任何单个字符,除了换行符 `\n`。 + +* :匹配前面的元素零次或多次。例如,a* 匹配零个或多个 'a'。 + ++ :匹配前面的元素一次或多次。例如,a+ 匹配一个或多个 'a'。 + +? :匹配前面的元素零次或一次。例如,a? 匹配零个或一个 'a'。 + +[] :用于定义字符集合。例如,[aeiou] 匹配任何一个元音字母。 + +[^] :在字符集合中使用 ^ 表示否定,匹配不在集合内的字符。例如,[^0-9] 匹配任何非数字字符。 + +() :用于分组表达式,可以对其应用量词。例如,(ab)+ 匹配连续的 'ab'。 + +| :用于表示或逻辑。例如,a|b 匹配 'a' 或 'b'。 + +^ :匹配字符串的开头。例如,^abc 匹配以 'abc' 开头的字符串。 + +$ :匹配字符串的结尾。例如,xyz$ 匹配以 'xyz' 结尾的字符串。 + +\ :用于转义特殊字符,使其变成普通字符。例如,\\. 匹配句点字符 '.'。 + +\s:匹配任何空白字符,包括空格、制表符、换行符等。 -## 最佳实践 +\d:匹配任何数字字符,相当于 [0-9]。 +\w:匹配任何单词字符,包括字母、数字和下划线,相当于 [a-zA-Z0-9_]。 +``` \ No newline at end of file diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md index bc57f8d567acd..ca8b0230e0900 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md @@ -28,129 +28,156 @@ under the License. -## 描述 -该语句创建 SQL 阻止规则,该功能可用于限制任何 sql 语句(包括 DDL 和 DML 语句)。 +## 描述 -支持按用户配置 SQL 黑名单: +该语句用于创建 SQL 阻止规则 -- 通过正则匹配的方式拒绝指定 SQL -- 通过设置 partition_num, tablet_num, cardinality, 检查一个查询是否达到其中一个限制 - - partition_num, tablet_num, cardinality 可以一起设置,一旦一个查询达到其中一个限制,查询将会被拦截 +SQL_BLOCK_RULE 可用于限制用户执行某些操作,比如避免扫描大量数据等 -语法: +## 语法 ```sql -CREATE SQL_BLOCK_RULE rule_name -[PROPERTIES ("key"="value", ...)]; +CREATE SQL_BLOCK_RULE +PROPERTIES ( + -- property + + -- Additional properties + [ , ... ] + ) ``` -参数说明: +## 必选参数 + +**1. ``** + +> 规则的名字 + +**2. ``** + +> 规则的属性,可以分为三类:执行 SQL 类,扫描限制类和开关类。 +> +> 执行 SQL 类和扫描限制类是互斥的,也就是说一个 SQL_BLOCK_RULE 只能限制其中一种。 +> +> **执行 SQL 类:** +> +>包含两种,分别代表正则匹配和精准匹配,两种只能选择其中一个。 +> +> - sql:匹配规则 (基于正则匹配,特殊字符需要转译,如`select *`使用`select \\*`),用户在执行 sql 时,系统会把这里设置的 sql 作为正则对用户提交的 sql 进行匹配,如果能匹配上,会阻止 sql 的运行。 +> +> - sqlHash: sql 的 md5 摘要值,这里主要是配合慢日志来使用,用户无需自己计算摘要值,例如慢日志中发现某个 sql 执行的很慢,可以在`fe.audit.log`复制`SqlHash`,创建 SQL_BLOCK_RULE 限制这个 sql 运行。 +> +> **扫描限制类** +> +> 用户发起查询的时候,查询优化器会计算出每张表需要扫描的 partition 数,tablet 数 和扫描的数据行数。以下属性分别对这三个数字进行限制,可以同时设置,也可以只限制部分 +> +> - partition_num: 一个表将扫描的最大 partition 数量 +> - tablet_num: 一个表将扫描的最大 tablet 数量 +> - cardinality: 一个表将扫描的数据行数 +> +> **开关类** +> +> - global:是否全局 (所有用户) 生效,默认为 false。不设置为 true 的话,需要通过 set property 对某个用户生效 +> - enable:是否开启阻止规则,默认为 true + +## 权限控制 + +执行此 SQL 命令的用户必须至少具有以下权限: -- sql:匹配规则 (基于正则匹配,特殊字符需要转译,如`select *`使用`select \\*`),可选,默认值为 "NULL", 最后不要带分号 -- sqlHash: sql hash 值,用于完全匹配,我们会在`fe.audit.log`打印这个值,可选,这个参数和 sql 只能二选一,默认值为 "NULL" -- partition_num: 一个扫描节点会扫描的最大 partition 数量,默认值为 0L -- tablet_num: 一个扫描节点会扫描的最大 tablet 数量,默认值为 0L -- cardinality: 一个扫描节点粗略的扫描行数,默认值为 0L -- global:是否全局 (所有用户) 生效,默认为 false -- enable:是否开启阻止规则,默认为 true +| 权限 | 对象 | 说明 | +| :---------------- | :------------- | :------------ | +| ADMIN_PRIV | 全局 | | ## 示例 -1. 创建一个名称为 test_rule 的阻止规则 +1. 创建不允许所有用户执行 `select * from order_analysis`的规则 + + ```sql + CREATE SQL_BLOCK_RULE test_rule + PROPERTIES( + "sql"="select \\* from order_analysis", + "global"="true", + "enable"="true" + ); + ``` - ```sql - CREATE SQL_BLOCK_RULE test_rule - PROPERTIES( - "sql"="select \\* from order_analysis", - "global"="false", - "enable"="true" - ); - ``` - 当我们去执行刚才我们定义在规则里的 sql 时就会返回异常错误,示例如下: + 当我们去执行刚才定义在规则里的 sql 时就会返回异常错误,示例如下: - ```sql - mysql> select * from order_analysis; - ERROR 1064 (HY000): errCode = 2, detailMessage = sql match regex sql block rule: order_analysis_rule - ``` + ```sql + mysql> select * from order_analysis; + ERROR 1064 (HY000): errCode = 2, detailMessage = sql match regex sql block rule: order_analysis_rule + ``` -2. 创建 test_rule2,将最大扫描的分区数量限制在 30 个,最大扫描基数限制在 100 亿行,示例如下: +2. 创建不允许对同一个表扫描的分区数量超过 30 并且查询的数据量不能超过 100 亿行 ```sql CREATE SQL_BLOCK_RULE test_rule2 PROPERTIES ( - "partition_num" = "30", - "cardinality" = "10000000000", - "global" = "false", - "enable" = "true" + "partition_num" = "30", + "cardinality" = "10000000000", + "global" = "true", + "enable" = "true" ); - Query OK, 0 rows affected (0.01 sec) - ``` + ``` +3. SQL 的匹配是基于正则的,如果想匹配更多模式的 SQL 需要写相应的正则,比如忽略 SQL 中空格,还有 order 开头的表都不能查询,示例如下: -3. 创建包含特殊字符的 SQL BLOCK RULE,正则表达式中 ( 和 ) 符号是特殊符号,所以需要转义,示例如下: - ```sql CREATE SQL_BLOCK_RULE test_rule3 - PROPERTIES - ( - "sql" = "select count\\(1\\) from db1.tbl1" - ); - CREATE SQL_BLOCK_RULE test_rule4 - PROPERTIES - ( - "sql" = "select \\* from db1.tbl1" + PROPERTIES( + "sql"="\\s*select\\s*\\*\\s*from order_\\w*\\s*", + "global"="true", + "enable"="true" ); ``` - -4. SQL_BLCOK_RULE 中,SQL 的匹配是基于正则的,如果想匹配更多模式的 SQL 需要写相应的正则,比如忽略 SQL -中空格,还有 order_ 开头的表都不能查询,示例如下: - - ```sql - CREATE SQL_BLOCK_RULE test_rule4 - PROPERTIES( - "sql"="\\s*select\\s*\\*\\s*from order_\\w*\\s*", - "global"="false", - "enable"="true" - ); - ``` - -### 附录 -常用正则表达式如下: -> . :匹配任何单个字符,除了换行符 `\n`。 -> -> * :匹配前面的元素零次或多次。例如,a* 匹配零个或多个 'a'。 -> -> + :匹配前面的元素一次或多次。例如,a+ 匹配一个或多个 'a'。 -> -> ? :匹配前面的元素零次或一次。例如,a? 匹配零个或一个 'a'。 -> -> [] :用于定义字符集合。例如,[aeiou] 匹配任何一个元音字母。 -> -> [^] :在字符集合中使用 ^ 表示否定,匹配不在集合内的字符。例如,[^0-9] 匹配任何非数字字符。 -> -> () :用于分组表达式,可以对其应用量词。例如,(ab)+ 匹配连续的 'ab'。 -> -> | :用于表示或逻辑。例如,a|b 匹配 'a' 或 'b'。 -> -> ^ :匹配字符串的开头。例如,^abc 匹配以 'abc' 开头的字符串。 -> -> $ :匹配字符串的结尾。例如,xyz$ 匹配以 'xyz' 结尾的字符串。 -> -> \ :用于转义特殊字符,使其变成普通字符。例如,\\. 匹配句点字符 '.'。 -> -> \s:匹配任何空白字符,包括空格、制表符、换行符等。 -> -> \d:匹配任何数字字符,相当于 [0-9]。 -> -> \w:匹配任何单词字符,包括字母、数字和下划线,相当于 [a-zA-Z0-9_]。 +4. 创建只针对部分用户的规则 + + ```sql + CREATE SQL_BLOCK_RULE test_rule4 + PROPERTIES( + "sql"="select \\* from order_analysis", + "global"="false", + "enable"="true" + ); + ``` + + 将规则作用于用户 jack + + ```sql + SET PROPERTY FOR 'jack' 'sql_block_rules' = 'test_rule4'; + ``` -## 关键词 +## 其它 + +常用正则表达式如下: ```text -CREATE, SQL_BLCOK_RULE -``` +. :匹配任何单个字符,除了换行符 `\n`。 + +* :匹配前面的元素零次或多次。例如,a* 匹配零个或多个 'a'。 + ++ :匹配前面的元素一次或多次。例如,a+ 匹配一个或多个 'a'。 + +? :匹配前面的元素零次或一次。例如,a? 匹配零个或一个 'a'。 + +[] :用于定义字符集合。例如,[aeiou] 匹配任何一个元音字母。 + +[^] :在字符集合中使用 ^ 表示否定,匹配不在集合内的字符。例如,[^0-9] 匹配任何非数字字符。 + +() :用于分组表达式,可以对其应用量词。例如,(ab)+ 匹配连续的 'ab'。 + +| :用于表示或逻辑。例如,a|b 匹配 'a' 或 'b'。 + +^ :匹配字符串的开头。例如,^abc 匹配以 'abc' 开头的字符串。 + +$ :匹配字符串的结尾。例如,xyz$ 匹配以 'xyz' 结尾的字符串。 + +\ :用于转义特殊字符,使其变成普通字符。例如,\\. 匹配句点字符 '.'。 + +\s:匹配任何空白字符,包括空格、制表符、换行符等。 -### 最佳实践 +\d:匹配任何数字字符,相当于 [0-9]。 +\w:匹配任何单词字符,包括字母、数字和下划线,相当于 [a-zA-Z0-9_]。 +``` \ No newline at end of file diff --git a/versioned_docs/version-2.1/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md b/versioned_docs/version-2.1/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md index 3cbf57e4a2c3a..0318e11390023 100644 --- a/versioned_docs/version-2.1/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md +++ b/versioned_docs/version-2.1/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md @@ -28,127 +28,159 @@ under the License. ## Description -This statement creates a SQL blocking rule. it can restrict any kind of sql statements(no matter DDL or DML statement). +This statement is used to create an SQL block rule. -Supports configuring SQL blacklists by user: +SQL_BLOCK_RULE can be used to restrict users from performing certain operations, such as avoiding scanning large amounts of data. -- Refuse to specify SQL by regular matching -- Check if a sql reaches one of these limits by setting partition_num, tablet_num, cardinality - - partition_num, tablet_num, cardinality can be set together, once a query reaches one of these limits, the query will be intercepted - -grammar: +## Syntax ```sql -CREATE SQL_BLOCK_RULE rule_name -[PROPERTIES ("key"="value", ...)]; +CREATE SQL_BLOCK_RULE +PROPERTIES ( + -- property + + -- Additional properties + [ , ... ] + ) ``` -Parameter Description: +## Required Parameters + +**1. ``** + +> The name of the rule. + +**2. ``** + +> The properties of the rule can be divided into three categories: SQL execution, scan limitation, and switch. +> +> The SQL execution and scan limitation categories are mutually exclusive, meaning that an SQL block rule can only restrict one of them. +> +> +> **SQL Execution Category** + +> There are two types, representing regular expression matching and exact matching, respectively. Only one of them can be chosen. +> +> - sql: The matching rule (based on regular expression matching, special characters need to be escaped, for example, `select *` should be written as `select \\*`). When a user executes an SQL statement, the system will use the SQL set here as a regular expression to match the SQL submitted by the user. If it matches, the execution of the SQL will be blocked. +> - sqlHash: The MD5 hash value of the SQL. This is mainly used in conjunction with slow logs. Users do not need to calculate the hash value themselves. For example, if a slow log shows that a particular SQL is running slowly, you can copy the `SqlHash` from `fe.audit.log` and create an SQL_BLOCK_RULE to restrict the execution of this SQL. +> +> **Scan Limitation Category** +> When a user initiates a query, the query optimizer will calculate the number of partitions, tablets, and rows of data that need to be scanned for each table. The following properties can be used to limit these three numbers, either all at once or just some of them. +> - partition_num: The maximum number of partitions that a table will scan. +> - tablet_num: The maximum number of tablets that a table will scan. +> - cardinality: The number of rows of data that a table will scan. +> +> **Switch Category** +> +> - global: Whether the rule is effective for all users. The default is false. If it is not set to true, the rule needs to be applied to a specific user through the `set property` command. +> - enable: Whether the blocking rule is enabled. The default is true. + +## Access Control Requirements -- sql: matching rule (based on regular matching, special characters need to be translated,for example`select *`use`select \\*`), optional, the default value is "NULL" -- sqlHash: sql hash value, used for exact matching, we will print this value in `fe.audit.log`, optional, this parameter and sql can only be selected one, the default value is "NULL" -- partition_num: the maximum number of partitions a scan node will scan, the default value is 0L -- tablet_num: The maximum number of tablets that a scanning node will scan, the default value is 0L -- cardinality: the rough scan line number of a scan node, the default value is 0L -- global: Whether to take effect globally (all users), the default is false -- enable: whether to enable blocking rules, the default is true +The user executing this SQL command must have at least the following permissions: -## Examples +| Privilege | Object | Notes | +| ------------ | ------ | ----- | +| ADMIN_PRIV | Global | | -1. Create a block rule named test_rule +## Example - ```sql - CREATE SQL_BLOCK_RULE test_rule - PROPERTIES( - "sql"="select \\* from order_analysis", - "global"="false", - "enable"="true" - ); - ``` +1. Create a rule that prevents all users from executing `select * from order_analysis` - >Notes: - > - >That the sql statement here does not end with a semicolon - - When we execute the sql we just defined in the rule, an exception error will be returned. The example is as follows: - - ```sql - select * from order_analysis; - ERROR 1064 (HY000): errCode = 2, detailMessage = sql match regex sql block rule: order_analysis_rule - ``` + ```sql + CREATE SQL_BLOCK_RULE test_rule + PROPERTIES( + "sql"="select \\* from order_analysis", + "global"="true", + "enable"="true" + ); + ``` + When we execute the SQL defined in the rule, it will return an exception error, as shown below: -2. Create test_rule2, limit the maximum number of scanned partitions to 30, and limit the maximum scan base to 10 billion rows. The example is as follows: ```sql - CREATE SQL_BLOCK_RULE test_rule2 - PROPERTIES ( - "partition_num" = "30", - "cardinality" = "10000000000", - "global" = "false", - "enable" = "true" - ); + mysql> select * from order_analysis; + ERROR 1064 (HY000): errCode = 2, detailMessage = sql match regex sql block rule: order_analysis_rule ``` -3. Create SQL BLOCK RULE with special chars - - ```sql - CREATE SQL_BLOCK_RULE test_rule3 - PROPERTIES - ( - "sql" = "select count\\(1\\) from db1.tbl1" - ); - CREATE SQL_BLOCK_RULE test_rule4 - PROPERTIES - ( - "sql" = "select \\* from db1.tbl1" - ); - ``` -4. In SQL_BLOCK_RULE, SQL matching is based on regular expressions. If want to match more patterns of SQL, need to write the corresponding regex. For example, to ignore spaces in SQL and not query tables that start with 'order_', as shown below: - - ```sql - CREATE SQL_BLOCK_RULE test_rule4 - PROPERTIES( - "sql"="\\s*select\\s*\\*\\s*from order_\\w*\\s*", + +2. Create a rule that prevents scanning more than 30 partitions of the same table and restricts the query data volume to no more than 10 billion rows + + + ```sql + CREATE SQL_BLOCK_RULE test_rule2 + PROPERTIES + ( + "partition_num" = "30", + "cardinality" = "10000000000", + "global" = "true", + "enable" = "true" + ); + ``` + +3. SQL matching is based on regular expressions. If you want to match more patterns of SQL, you need to write the corresponding regular expressions. For example, ignoring spaces in SQL and preventing queries on tables starting with "order", as shown below: + + + ```sql + CREATE SQL_BLOCK_RULE test_rule3 + PROPERTIES( + "sql"="\\s*select\\s*\\*\\s*from order_\\w*\\s*", + "global"="true", + "enable"="true" + ); + ``` + +4. Create a rule that is only applicable to specific users + + + ```sql + CREATE SQL_BLOCK_RULE test_rule4 + PROPERTIES( + "sql"="select \\* from order_analysis", "global"="false", "enable"="true" - ); - ``` + ); + ``` + + Apply the rule to user jack -### APPENDIX -Here are some commonly used regular expressions: -> . :Matches any single character except for a newline character \n. -> -> * :Matches the preceding element zero or more times. For example, a matches zero or more 'a'. -> -> + :Matches the preceding element one or more times. For example, a+ matches one or more 'a'. -> -> ? :Matches the preceding element zero or one time. For example, a? matches zero or one 'a'. -> -> [] :Used to define a character set. For example, [aeiou] matches any one vowel letter. -> -> [^] :In a character set, use ^ to indicate negation, matching characters that are not in the set. For example, [^0-9] matches any non-digit character. -> -> () :Used for grouping expressions and applying quantifiers. For example, (ab)+ matches consecutive 'ab'. -> -> | :Represents logical OR. For example, a|b matches 'a' or 'b'. -> -> ^ :Matches the beginning of a string. For example, ^abc matches a string that starts with 'abc'. -> -> $ :Matches the end of a string. For example, xyz$ matches a string that ends with 'xyz'. -> -> \ :Used to escape special characters to treat them as ordinary characters. For example, \\. matches the period character '.'. -> -> \s :Matches any whitespace character, including spaces, tabs, newline characters, etc. -> -> \d :Matches any digit character, equivalent to [0-9]. -> -> \w :Matches any word character, including letters, digits, and underscores, equivalent to [a-zA-Z0-9_]. -## Keywords + ```sql + SET PROPERTY FOR 'jack' 'sql_block_rules' = 'test_rule4'; + ``` + +## Others + +Common regular expressions are as follows: + +TextCopy ```text -CREATE, SQL_BLCOK_RULE -``` +. : Matches any single character except the newline character `\n`. + +* : Matches the preceding element zero or more times. For example, a* matches zero or more 'a's. + ++ : Matches the preceding element one or more times. For example, a+ matches one or more 'a's. + +? : Matches the preceding element zero or one time. For example, a? matches zero or one 'a'. + +[] : Defines a character set. For example, [aeiou] matches any vowel. + +[^] : When used in a character set, ^ indicates negation, matching characters not in the set. For example, [^0-9] matches any non-digit character. + +() : Groups expressions, allowing quantifiers to be applied to them. For example, (ab)+ matches consecutive 'ab's. + +| : Indicates logical OR. For example, a|b matches 'a' or 'b'. + +^ : Matches the beginning of a string. For example, ^abc matches strings starting with 'abc'. + +$ : Matches the end of a string. For example, xyz$ matches strings ending with 'xyz'. + +\ : Escapes special characters, making them ordinary characters. For example, \\. matches the period character '.'. + +\s : Matches any whitespace character, including spaces, tabs, newlines, etc. -## Best Practice +\d : Matches any digit character, equivalent to [0-9]. +\w : Matches any word character, including letters, digits, and underscores, equivalent to [a-zA-Z0-9_]. +``` \ No newline at end of file diff --git a/versioned_docs/version-3.0/sql-manual/sql-statements/data-governance/ALTER-SQL_BLOCK_RULE.md b/versioned_docs/version-3.0/sql-manual/sql-statements/data-governance/ALTER-SQL_BLOCK_RULE.md index 9c29f01ac464c..17271ab1728e9 100644 --- a/versioned_docs/version-3.0/sql-manual/sql-statements/data-governance/ALTER-SQL_BLOCK_RULE.md +++ b/versioned_docs/version-3.0/sql-manual/sql-statements/data-governance/ALTER-SQL_BLOCK_RULE.md @@ -25,8 +25,6 @@ under the License. --> - - ## Description This statement is used to modify an SQL block rule. diff --git a/versioned_docs/version-3.0/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md b/versioned_docs/version-3.0/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md index c3fd0691ea599..98c35823e0046 100644 --- a/versioned_docs/version-3.0/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md +++ b/versioned_docs/version-3.0/sql-manual/sql-statements/data-governance/CREATE-SQL_BLOCK_RULE.md @@ -27,127 +27,161 @@ under the License. ## Description -This statement creates a SQL blocking rule. it can restrict any kind of sql statements(no matter DDL or DML statement). +This statement is used to create an SQL block rule. -Supports configuring SQL blacklists by user: +SQL_BLOCK_RULE can be used to restrict users from performing certain operations, such as avoiding scanning large amounts of data. -- Refuse to specify SQL by regular matching -- Check if a sql reaches one of these limits by setting partition_num, tablet_num, cardinality - - partition_num, tablet_num, cardinality can be set together, once a query reaches one of these limits, the query will be intercepted - -grammar: +## Syntax ```sql -CREATE SQL_BLOCK_RULE rule_name -[PROPERTIES ("key"="value", ...)]; +CREATE SQL_BLOCK_RULE +PROPERTIES ( + -- property + + -- Additional properties + [ , ... ] + ) ``` -Parameter Description: +## Required Parameters + +**1. ``** + +> The name of the rule. + + +**2. ``** + +> The properties of the rule can be divided into three categories: SQL execution, scan limitation, and switch. +> +> The SQL execution and scan limitation categories are mutually exclusive, meaning that an SQL block rule can only restrict one of them. +> +> +> **SQL Execution Category** +> +> There are two types, representing regular expression matching and exact matching, respectively. Only one of them can be chosen. +> +> - sql: The matching rule (based on regular expression matching, special characters need to be escaped, for example, `select *` should be written as `select \\*`). When a user executes an SQL statement, the system will use the SQL set here as a regular expression to match the SQL submitted by the user. If it matches, the execution of the SQL will be blocked. +> - sqlHash: The MD5 hash value of the SQL. This is mainly used in conjunction with slow logs. Users do not need to calculate the hash value themselves. For example, if a slow log shows that a particular SQL is running slowly, you can copy the `SqlHash` from `fe.audit.log` and create an SQL_BLOCK_RULE to restrict the execution of this SQL. +> +> **Scan Limitation Category** +> When a user initiates a query, the query optimizer will calculate the number of partitions, tablets, and rows of data that need to be scanned for each table. The following properties can be used to limit these three numbers, either all at once or just some of them. +> - partition_num: The maximum number of partitions that a table will scan. +> - tablet_num: The maximum number of tablets that a table will scan. +> - cardinality: The number of rows of data that a table will scan. +> +> **Switch Category** +> +> - global: Whether the rule is effective for all users. The default is false. If it is not set to true, the rule needs to be applied to a specific user through the `set property` command. +> - enable: Whether the blocking rule is enabled. The default is true. + + +## Access Control Requirements -- sql: matching rule (based on regular matching, special characters need to be translated,for example`select *`use`select \\*`), optional, the default value is "NULL" -- sqlHash: sql hash value, used for exact matching, we will print this value in `fe.audit.log`, optional, this parameter and sql can only be selected one, the default value is "NULL" -- partition_num: the maximum number of partitions a scan node will scan, the default value is 0L -- tablet_num: The maximum number of tablets that a scanning node will scan, the default value is 0L -- cardinality: the rough scan line number of a scan node, the default value is 0L -- global: Whether to take effect globally (all users), the default is false -- enable: whether to enable blocking rules, the default is true +The user executing this SQL command must have at least the following permissions: + +| Privilege | Object | Notes | +| ------------ | ------ | ----- | +| ADMIN_PRIV | Global | | ## Example -1. Create a block rule named test_rule +1. Create a rule that prevents all users from executing `select * from order_analysis` + + ```sql + CREATE SQL_BLOCK_RULE test_rule + PROPERTIES( + "sql"="select \\* from order_analysis", + "global"="true", + "enable"="true" + ); + ``` + + When we execute the SQL defined in the rule, it will return an exception error, as shown below: + + + ```sql + mysql> select * from order_analysis; + ERROR 1064 (HY000): errCode = 2, detailMessage = sql match regex sql block rule: order_analysis_rule + ``` + +2. Create a rule that prevents scanning more than 30 partitions of the same table and restricts the query data volume to no more than 10 billion rows - ```sql - CREATE SQL_BLOCK_RULE test_rule - PROPERTIES( - "sql"="select \\* from order_analysis", - "global"="false", - "enable"="true" - ); - ``` - >Notes: - > - >That the sql statement here does not end with a semicolon - - When we execute the sql we just defined in the rule, an exception error will be returned. The example is as follows: - - ```sql - select * from order_analysis; - ERROR 1064 (HY000): errCode = 2, detailMessage = sql match regex sql block rule: order_analysis_rule - ``` + ```sql + CREATE SQL_BLOCK_RULE test_rule2 + PROPERTIES + ( + "partition_num" = "30", + "cardinality" = "10000000000", + "global" = "true", + "enable" = "true" + ); + ``` +3. SQL matching is based on regular expressions. If you want to match more patterns of SQL, you need to write the corresponding regular expressions. For example, ignoring spaces in SQL and preventing queries on tables starting with "order", as shown below: -2. Create test_rule2, limit the maximum number of scanned partitions to 30, and limit the maximum scan base to 10 billion rows. The example is as follows: ```sql - CREATE SQL_BLOCK_RULE test_rule2 - PROPERTIES ( - "partition_num" = "30", - "cardinality" = "10000000000", - "global" = "false", - "enable" = "true" - ); + CREATE SQL_BLOCK_RULE test_rule3 + PROPERTIES( + "sql"="\\s*select\\s*\\*\\s*from order_\\w*\\s*", + "global"="true", + "enable"="true" + ); ``` -3. Create SQL BLOCK RULE with special chars - - ```sql - CREATE SQL_BLOCK_RULE test_rule3 - PROPERTIES - ( - "sql" = "select count\\(1\\) from db1.tbl1" - ); - CREATE SQL_BLOCK_RULE test_rule4 - PROPERTIES - ( - "sql" = "select \\* from db1.tbl1" - ); - ``` -4. In SQL_BLOCK_RULE, SQL matching is based on regular expressions. If want to match more patterns of SQL, need to write the corresponding regex. For example, to ignore spaces in SQL and not query tables that start with 'order_', as shown below: - - ```sql - CREATE SQL_BLOCK_RULE test_rule4 - PROPERTIES( - "sql"="\\s*select\\s*\\*\\s*from order_\\w*\\s*", + +4. Create a rule that is only applicable to specific users + + + ```sql + CREATE SQL_BLOCK_RULE test_rule4 + PROPERTIES( + "sql"="select \\* from order_analysis", "global"="false", "enable"="true" - ); - ``` + ); + ``` -### APPENDIX -Here are some commonly used regular expressions: -> . :Matches any single character except for a newline character \n. -> -> * :Matches the preceding element zero or more times. For example, a matches zero or more 'a'. -> -> + :Matches the preceding element one or more times. For example, a+ matches one or more 'a'. -> -> ? :Matches the preceding element zero or one time. For example, a? matches zero or one 'a'. -> -> [] :Used to define a character set. For example, [aeiou] matches any one vowel letter. -> -> [^] :In a character set, use ^ to indicate negation, matching characters that are not in the set. For example, [^0-9] matches any non-digit character. -> -> () :Used for grouping expressions and applying quantifiers. For example, (ab)+ matches consecutive 'ab'. -> -> | :Represents logical OR. For example, a|b matches 'a' or 'b'. -> -> ^ :Matches the beginning of a string. For example, ^abc matches a string that starts with 'abc'. -> -> $ :Matches the end of a string. For example, xyz$ matches a string that ends with 'xyz'. -> -> \ :Used to escape special characters to treat them as ordinary characters. For example, \\. matches the period character '.'. -> -> \s :Matches any whitespace character, including spaces, tabs, newline characters, etc. -> -> \d :Matches any digit character, equivalent to [0-9]. -> -> \w :Matches any word character, including letters, digits, and underscores, equivalent to [a-zA-Z0-9_]. + Apply the rule to user jack + + + ```sql + SET PROPERTY FOR 'jack' 'sql_block_rules' = 'test_rule4'; + ``` + +## Others + +Common regular expressions are as follows: -## Keywords +TextCopy ```text -CREATE, SQL_BLCOK_RULE -``` +. : Matches any single character except the newline character `\n`. + +* : Matches the preceding element zero or more times. For example, a* matches zero or more 'a's. + ++ : Matches the preceding element one or more times. For example, a+ matches one or more 'a's. + +? : Matches the preceding element zero or one time. For example, a? matches zero or one 'a'. + +[] : Defines a character set. For example, [aeiou] matches any vowel. + +[^] : When used in a character set, ^ indicates negation, matching characters not in the set. For example, [^0-9] matches any non-digit character. + +() : Groups expressions, allowing quantifiers to be applied to them. For example, (ab)+ matches consecutive 'ab's. + +| : Indicates logical OR. For example, a|b matches 'a' or 'b'. + +^ : Matches the beginning of a string. For example, ^abc matches strings starting with 'abc'. + +$ : Matches the end of a string. For example, xyz$ matches strings ending with 'xyz'. + +\ : Escapes special characters, making them ordinary characters. For example, \\. matches the period character '.'. + +\s : Matches any whitespace character, including spaces, tabs, newlines, etc. -## Best Practice +\d : Matches any digit character, equivalent to [0-9]. +\w : Matches any word character, including letters, digits, and underscores, equivalent to [a-zA-Z0-9_]. +``` \ No newline at end of file