查看当前时间的命令datemrchip@ubuntu:~$ date Wed Oct 30 12:06:46 AM UTC 2024修改时区获得正确的时间mrchip@ubuntu:~$ cat /etc/localtime TZif2UTCTZif2UTC UTC0 mrchip@ubuntu:~$ sudo rm /etc/localtime mrchip@ubuntu:~$ sudo ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime mrchip@ubuntu:~$ date Wed Oct 30 08:10:25 AM CST 2024
之前写过mosuqitto编译和使用相关的博文，本文将介绍一个可以在生产环境使用的mosquitto配置过程，引入了更多的权限和安全管理。这次实验博主是在阿里云的ECS上完成的，没有云服务器的小伙伴可以在本地安装一个Ubuntu 22.04的虚拟机或者看看腾讯云或者阿里云有没有优惠的云服务器。获取mosquitto的源代码和mosquitto-go-auth的源代码wget https://mosquitto.org/files/source/mosquitto-2.0.18.tar.gz git clone https://github.com/iegomez/mosquitto-go-auth.git编译安装mosquitto编译安装mosquitto程序apt install libssl-dev libcjson-dev tar -zxf mosquitto-2.0.18.tar.gz cd mosquitto-2.0.18/ make make install groupadd mosquitto useradd -s /sbin/nologin mosquitto -g mosquitto -d /var/lib/mosquitto mkdir -p /var/log/mosquitto/ /var/lib/mosquitto/ chown -R mosquitto:mosquitto /var/log/mosquitto/ chown -R mosquitto:mosquitto /var/lib/mosquitto/编写配置文件，修改监听地址等等cd /etc/mosquitto/ cp mosquitto.conf.example mosquitto.conf vim mosquitto.conf测试用的配置allow_anonymous true listener 1883 0.0.0.0然后运行进行测试，查看客户端能否连接mosquitto -c /etc/mosquitto/mosquitto.conf连上之后我们来写一个服务配置文件，使得systemd能够管理服务的启动vim /etc/systemd/system/mosquitto.service填入以下内容[Unit] Description=Mosquitto MQTT v3.1/v5 server Wants=network.target Documentation=http://mosquitto.org/documentation/ [Service] Type=simple User=mosquitto Group=mosquitto ExecStart=/usr/local/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf Restart=on-failure SyslogIdentifier=Mosquitto [Install] WantedBy=multi-user.targetsystemctl start mosquitto.service systemctl enable mosquitto.service编译配置go-auth插件接下来就可以编译mosquitto-go-auth插件了，首先确保自己安装了go-1.18以上的版本cd mosquitto-go-auth make构建完成后可以看到目录下有一个go-auth.so和pw可执行文件分别是mosquitto的插件和密码生成工具，将这两个工具拷贝到/etc/mosquitto/conf.dcp go-auth.so /etc/mosquitto/conf.d/ cp pw /etc/mosquitto/conf.d编写一个go-auth插件的配置文件vim /etc/mosquitto/conf.d/go-auth.conf填入以下内容配置通过文件来授权auth_plugin /etc/mosquitto/conf.d/go-auth.so auth_opt_backends files auth_opt_disable_superuser true auth_opt_files_password_path /etc/mosquitto/conf.d/password_file auth_opt_files_acl_path /etc/mosquitto/conf.d/acl_file auth_opt_hasher_salt_size 16 # salt bytes length auth_opt_hasher_iterations 100000 # number of iterations auth_opt_hasher_keylen 64 # key length auth_opt_hasher_algorithm sha512 # hashing algorithm, either sha512 (default) or sha256 auth_opt_hasher_salt_encoding # salt encoding, either base64 (default) or utf-8 auth_opt_hasher pbkdf2 auth_opt_log_dest file auth_opt_log_file /var/log/mosquitto/mosquitto.log auth_opt_log_level debug并且生成一个配置文件/etc/mosquitto/conf.d/acl_fileuser test1 topic write test/topic/1 topic read test/topic/2 user test2 topic read test/topic/+通过刚才拷贝的工具生成加密的密码./pw -p <your password>然后将生成的字符串写入到/etc/mosquitto/conf.d/password_file中，其中用户名test1的密码是test，test2的密码是mosquittotest1:PBKDF2$sha512$100000$fAdA5klpGch9lRCmwM2Ozw==$fxkXYWi0QlFtk+oYdMHYZxBQSNtJ05ogHxfKmza5CpUDGL6+MysG3ier8hFl6BsAsE3t1Tq6Y3JkEzn9mhlLgQ== test2:PBKDF2$sha512$100000$JecNIqpn7hX8dueP3gQsGQ==$1nYeN8L9V96/C9yAUz7FokqzQq1DpyjBRcxvR1fauCsie+q9cb4dxcFvPbi4V6JgiNHGRP1Zn2vQp3KiTLlRgA==修改mosquitto.conf的配置以支持插件的配置include_dir /etc/mosquitto/conf.d allow_anonymous false重启服务以后，我们发现必须需要用户名密码正确才可以连接了，并且对主题的独写权限也有了区分。配置Postgres和Redis缓存安装Redis作为缓存apt update apt install redis配置redis的密码，vim /etc/redis/redis.conf，增加以下行requirepass mosquitto启动redissystemctl start redis-server systemctl enable redis-server安装Postgresqlapt install postgresql配置数据库sudo -u postgres psql执行以下sql语句创建数据库create user mosquitto with login password 'mosquitto'; create database mosquitto with owner mosquitto; \c mosquitto; GRANT ALL PRIVILEGES ON DATABASE mosquitto to mosquitto; GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO mosquitto; create table tb_user( id bigserial primary key, username character varying (100) not null, password_hash character varying (200) not null, is_admin boolean not null); create table tb_acl( id bigserial primary key, user_id bigint not null references tb_user on delete cascade, topic character varying (200) not null, rw int not null); insert into tb_user (username, password_hash, is_admin) VALUES ('test1', 'PBKDF2$sha512$100000$fAdA5klpGch9lRCmwM2Ozw==$fxkXYWi0QlFtk+oYdMHYZxBQSNtJ05ogHxfKmza5CpUDGL6+MysG3ier8hFl6BsAsE3t1Tq6Y3JkEzn9mhlLgQ==', false); insert into tb_user (username, password_hash, is_admin) VALUES ('test2', 'PBKDF2$sha512$100000$JecNIqpn7hX8dueP3gQsGQ==$1nYeN8L9V96/C9yAUz7FokqzQq1DpyjBRcxvR1fauCsie+q9cb4dxcFvPbi4V6JgiNHGRP1Zn2vQp3KiTLlRgA==', false); insert into tb_acl (user_id, topic, rw) VALUES (1, 'test/topic/1', 2); insert into tb_acl (user_id, topic, rw) VALUES (1, 'test/topic/2', 1); insert into tb_acl (user_id, topic, rw) VALUES (2, 'test/topic/#', 1); 修改go-auth.confauth_plugin /etc/mosquitto/conf.d/go-auth.so auth_opt_backends postgres auth_opt_disable_superuser true auth_opt_files_password_path /etc/mosquitto/conf.d/password_file auth_opt_files_acl_path /etc/mosquitto/conf.d/acl_file auth_opt_hasher_salt_size 16 # salt bytes length auth_opt_hasher_iterations 100000 # number of iterations auth_opt_hasher_keylen 64 # key length auth_opt_hasher_algorithm sha512 # hashing algorithm, either sha512 (default) or sha256 auth_opt_hasher_salt_encoding # salt encoding, either base64 (default) or utf-8 auth_opt_hasher pbkdf2 auth_opt_log_dest file auth_opt_log_file /var/log/mosquitto/mosquitto.log auth_opt_log_level info auth_opt_cache true auth_opt_cache_type redis auth_opt_cache_reset true auth_opt_cache_refresh true auth_opt_auth_cache_seconds 30 auth_opt_acl_cache_seconds 30 auth_opt_auth_jitter_seconds 5 auth_opt_auth_jitter_seconds 5 auth_opt_cache_host 127.0.0.1 auth_opt_cache_port 6379 auth_opt_cache_password mosquitto auth_opt_cache_db 3 auth_opt_pg_host 127.0.0.1 auth_opt_pg_port 5432 auth_opt_pg_dbname mosquitto auth_opt_pg_user mosquitto auth_opt_pg_password mosquitto auth_opt_pg_sslmode disable auth_opt_pg_connect_tries 5 auth_opt_pg_userquery select password_hash from "tb_user" where username = $1 limit 1 auth_opt_pg_superquery select count(*) from "tb_user" where username = $1 and is_admin = true auth_opt_pg_aclquery select acl.topic FROM tb_acl acl inner join tb_user u ON u.id = acl.user_id WHERE u.username = $1 AND acl.rw >= $重启mosquitto即可生效参考链接mosquitto-go-auth插件Redis相关的配置在Ubuntu22.04上安装Redispostgresql配置改变表格权限benchmark
目的早两天写了一篇S3简单上传文件的小工具，知乎上看到了一个问题问如何实现显示MINIO上传进度，因此拓展一下这个小工具能够在上传大文件时显示进度。完整代码托管于Github：mrchipset/simple-wpf实现方式先通过Xaml编写一个包含上传进度条的小界面。具体内容就不赘述了，可以参考这篇文章为了得到上传进度就不能再简单地使用PutObjectRequest 进行上传需要使用S3中TransferUtility 提供的高等级API进行上传。然后创建一个TransferUtilityUploadRequest 对象并绑定其UploadProgressEvent 事件以实现上传进度的监控具体的实现代码如下：private async Task<bool> UploadLargeFileAsync() { var credentials = new BasicAWSCredentials(_accessKey, _secretKey); var clientConfig = new AmazonS3Config { ForcePathStyle = true, ServiceURL = _endpoint, }; bool ret = true; using (var client = new AmazonS3Client(credentials, clientConfig)) { try { var fileTransferUtility = new TransferUtility(client); var uploadRequest = new TransferUtilityUploadRequest { BucketName = LargeBucket, FilePath = UploadLargeFile, Key = System.IO.Path.GetFileName(UploadLargeFile) }; uploadRequest.UploadProgressEvent += UploadRequest_UploadProgressEvent; await fileTransferUtility.UploadAsync(uploadRequest); } catch (FileNotFoundException e) { ret = false; this.Dispatcher.Invoke(new Action(() => this.statusLargeTxtBlk.Text = e.Message)); } catch (AmazonS3Exception e) { ret = false; if (e.ErrorCode != null && (e.ErrorCode.Equals("InvalidAccessKeyId") || e.ErrorCode.Equals("InvalidSecurity"))) { this.Dispatcher.Invoke(new Action(() => this.statusLargeTxtBlk.Text = "Please check the provided AWS Credentials")); } else { this.Dispatcher.Invoke(new Action(() => this.statusLargeTxtBlk.Text = $"An error occurred with the message '{e.Message}' when writing an object")); } } catch(Exception e) { this.Dispatcher.Invoke(new Action(() => this.statusLargeTxtBlk.Text = $"An error occurred with the message '{e.Message}' when writing an object")); } } return ret; } private void UploadRequest_UploadProgressEvent(object? sender, UploadProgressArgs e) { this.Dispatcher.Invoke((Action)(() => { this.uploadProgress.Value = e.TransferredBytes * 100 / e.TotalBytes ; })); }值得一提的时，在上传进度的事件处理函数中，由于我们通过异步方法执行上传函数，因此我们需要使用Dispatcher 来更新数据到UI 上。演示效果参考连接https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpu-upload-object.htmlhttps://www.xtigerkin.com/archives/96/
之前一直用docker的方式搭建Minio对象存储的方式使用，这次心血来潮想要自己在VPS上进行搭建。发现minio就一个可执行文件就完事了，不得不佩服golang大法静态编译就是好啊。那么剩下的就是写个systemd启动脚本来配置自动启动了。搭建minio服务下载minio的可执行文件wget https://dl.min.io/server/minio/release/linux-amd64/minio chmod +x minio先通过命令行运行./minio server --console-address ":9090" --address ":9000" /root/minio/data可以看到如下输出root@instance-jw4aEb7i:~/minio/app# ./minio server --console-address ":9090" --address ":9000" /root/minio/data Formatting 1st pool, 1 set(s), 1 drives per set. WARNING: Host local has more than 0 drives of set. A host failure will result in data becoming unavailable. WARNING: Detected default credentials 'minioadmin:minioadmin', we recommend that you change these values with 'MINIO_ROOT_USER' and 'MINIO_ROOT_PASSWORD' environment variables MinIO Object Storage Server Copyright: 2015-2023 MinIO, Inc. License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html> Version: RELEASE.2023-12-02T10-51-33Z (go1.21.4 linux/amd64) Status: 1 Online, 0 Offline. S3-API: http://172.16.10.133:9000 http://127.0.0.1:9000 RootUser: minioadmin RootPass: minioadmin Console: http://172.16.10.133:9090 http://127.0.0.1:9090 RootUser: minioadmin RootPass: minioadmin Command-line: https://min.io/docs/minio/linux/reference/minio-mc.html#quickstart $ mc alias set 'myminio' 'http://172.16.10.133:9000' 'minioadmin' 'minioadmin' Documentation: https://min.io/docs/minio/linux/index.html Warning: The standard parity is set to 0. This can lead to data loss.接下来我们简单体验一下MINIO由于我们没有设置用户密码，所以从运行信息中找到如下登录信息成功登录将看到如下画面我们按照提示创建存储桶并且随便传点东西上去修改存储桶的访问策略，使得上传的文件可以被公网访问创建一个Access Keys通过客户端访问数据通过客户端上传一张图片接下来去公网访问，输入MINIO的访问服务地址+存储桶和文件名称，就可以访问到刚上传的文件可以看到刚刚上传的图片编写systemd的service将minio的目录移动到/opt/minio 编写一个配置文件存储到/etc/minio/minio.conf ，后续的启动脚本中会用到MINIO_VOLUMES="/opt/minio/data" MINIO_OPTS="--console-address :9090 --address :9000" MINIO_ROOT_PASSWORD="minioadmin" MINIO_ROOT_USER="minioadmin"编写minio.service存储到/etc/systemd/system[Unit] Description=MINIO Document=https://docs.min.io Wants=network-online.target After=network-online.target AssertFileIsExecutable=/opt/minio/app/minio [Service] PermissionsStartOnly=true EnvironmentFile=/etc/minio/minio.conf ExecStartPre=/bin/bash -c "[ -n "${MINIO_VOLUMES}" ] || echo "Variable MINIO_VOLUMES not set in /etc/minio/minio.conf"" ExecStart=/opt/minio/app/minio server $MINIO_OPTS $MINIO_VOLUMES Restart=always LimitNOFILE=65536 TimeoutStopSec=infinity SekndSIGKILL=no StandardOutput=journal StandardError=inherit [Install] WantedBy=multi-user.target然后更新systemd配置缓存systemctl daemon-reload启动/停止/查询状态systemctl [start|stop|status] minio.service设置开机启动systemctl enable minio.service参考链接https://blog.csdn.net/liu\_51com/article/details/122475877https://www.cnblogs.com/revol/p/14561389.html
消息队列遥测传输（英语：Message Queuing Telemetry Transport，MQTT）是ISO 标准（ISO/IEC PRF 20922）下基于发布（Publish）/订阅（Subscribe）范式的消息协议，可视为“资料传递的桥梁”。它工作在TCP/IP协议族上，是为硬件性能低下的远程设备以及网络状况糟糕的情况下而设计的发布/订阅型消息协议。为此，它需要一个消息中间件（如HTTP），以解决当前繁重的资料传输协议。Eclipse Mosquitto 是一款开源（EPL/EDL 许可）消息代理程序，可实现 5.0、3.1.1 和 3.1 版 MQTT 协议。Mosquitto 重量轻，适用于从低功耗单板计算机到完整服务器的所有设备。MQTT 协议提供了一种使用发布/订阅模式进行消息传递的轻量级方法。这使其适用于物联网消息传递，如低功耗传感器或移动设备（如手机、嵌入式计算机或微控制器）。Mosquitto 项目还提供了一个用于实现 MQTT 客户端的 C 库，以及非常流行的 mosquitto_pub 和 mosquitto_sub 命令行 MQTT 客户端。本文将对在Alpine Linux发行版上通过apk包管理器对mosquitto服务器和客户端进行介绍更新apk仓库列表并安装mosquitto服务端和客户端apk update apk add mosquitto mosquitto-clients配置服务器编辑mosquitto.conf 修改配置主要找到listener的部分，修改监听ip使得外网可以连接，配置用户listener 1883 0.0.0.0 password_file /etc/mosquitto/pwfile acl_file /ect/mosquitto/aclfile创建一个新的用户adminmosquitto_passwd /etc/mosquitto/pwfile adminaclfile 访问权限配置文件# This affects access control for clients with no username. topic read $SYS/# # This only affects clients with username "roger". user roger topic foo/bar # This affects all clients. pattern write $SYS/broker/connection/%c/state user admin topic readwrite #pwfile，加密的用户名和密码存储文件roger:$6$clQ4Ocu312S0qWgl$Cv2wUxgEN73c6C6jlBkswqR4AkHsvDLWvtEXZZ8NpsBLgP1WAo/qA+WXcmEN/mjDNgdUwcxRAveqNMs2xUVQYA== sub_client:$6$U+qg0/32F0g2Fh+n$fBPSkq/rfNyEQ/TkEjRgwGTTVBpvNhKSyGShovH9KHewsvJ731tD5Zx26IHhR5RYCICt0L9qBW0/KK31UkCliw== pub_client:$6$vxQ89y+7WrsnL2yn$fSPMmEZn9TSrC8s/jaPmxJ9NijWpkP2e7bMJLz78JXR1vW2x8+T3FZ23byJA6xs5Mt+LeOybAHwcUv0OCl40rA== admin:$7$101$qDUNQWf2L3MJySTg$7hj4qH6+tzYhXrk+GO0UVlFvuVK4GNsXTU93064ki6r8Je8Y/eOAhiLvsQKG3j+HJp1cVRhhEzpWrAz9wg7tkA==启动服务端通过mosquitto 指令启动mosquitto -c /etc/mosquitto/mosquitto.conf通过rc-service 启动rc-service mosquitto start输出 * Caching service dependencies ... [ ok ] * Starting Mosquitto message broker ... [ ok ]设置开机启动rc-update add mosquitto客户端测试 mosquitto_sub -h 127.0.0.1 -p 1883 -u admin -P admin -t "#"在本机客户端订阅所有topic在远程电脑客户端发布测试消息感谢雨云的云计算平台支持参考链接https://blog.csdn.net/hxj0323/article/details/108409225https://blog.csdn.net/u012377333/article/details/69397124https://blog.csdn.net/weixin\_43025071/article/details/82502094https://answers.launchpad.net/mosquitto/+question/226477https://cloudbool.com/archive/mosquitto-basic-config.htmlhttps://blog.csdn.net/u012377333/article/details/69397124https://www.jianshu.com/p/1a8a5814b5a8
给大家介绍一个非常良心的云平台——雨云。雨云提供了虚拟主机，云服务器，裸金属服务器等云计算资源。其中值得一提的是其为雨云产品免费提供的CDN服务。即便不是雨云的云产品也可以通过自定义回源地址的方式配合流量包使用。而且流量包是可以通过积分签到进行兑换的，基本等于白*嫖。本文将已图文教程的方式进行介绍，要使用雨云CDN需要有一个雨云账号，没有账号的小伙伴可以点击这个链接注册说一下雨云CDN的官方宣传优势：1000G的高防对于雨云的云资源可以免费使用（外站可以通过积分兑换，基本也是免费使用）没有备案的域名也可以使用？（这点笔者也不懂什么原理）分步解说选择云产品中的CDN输入网站域名对于雨云产品，直接选择红框按钮，下拉找到对应产品id即可外站则填入源站的IP地址输入回源端口，有ssl支持的，选择443端口和https协议，如果没有则使用80端口和http注意：如果SSL配置不正确，可能导致CDN无法回源（选做）如使用SSL支持，请上传域名对应的SSL证书到雨云的域名服务平台认证刚才输入的域名的所有权，认证方式就是在DNS解析中增加一条challeng的txt记录上述步骤都完成后，新增一条DNS解析记录，将刚才填入的域名CNAME跳转到雨云CDN的域名上输入地址测试，如果CDN正常就会发现，服务器的IP是雨云的IP了而不是原有的服务器IP地址最后安利一下，如果大家需要云服务和CDN，可以点击下方链接进行注册，云产品新购8折，续费85折https://www.rainyun.com/Tigerkin_