最近在做云备份功能。包含上传和下载。网上的断点上传太复杂了,不稳定,缺少重试机制等。我用最简单的方式交会你们
步骤和流程:
1.先分块,把块分好!
总大小/每一块的值
2.md5是一块文件的md5还是
都要上传,获取字节数组的md5
3. 多并发处理网络请求
并行:CountDownLatch来监听结果
当使用多线程去下载或者上传时,由于多个线程互不干扰的执行,怎么判断所有的线程是否执行完毕呢?线程池没有提供这样的方法,那么只能自己去实现了。一般可以设置一个整形的标志位,初始化为0,当一个线程完成后就把这个标志位+1,然后判断标志位是否等于=任务的数量,等于就代表所有任务都执行完成了,但是这样感觉不是很优雅。java中提供了一个计数器,我们可以使用CountDownLatch来判断所有任务是否完成
串行:每次循环,他都给我返回,即使是成功的也是,那么20多块,用串行的方式,写同步请求,得到一个再用一个
4.进度计算 (已上传的块数*每个块数大小)
5.结束的判断!怎么知道全部上传完成了? 判断所有的块都上传成功了
6.重试机制 添加计算计算
7.取消上传,有回调吗?是不是应该给取消接口
okhttp的取消回调
8.断点怎么实现的: 一块一块,上传的块不会再次上传
9.进程不在,怎么知道失败的个数和数量?
由后台返回数据才行。给我返回分块的总数,已经完成的个数!还有每一个的编号!
/**
*分片上传
*/
fun multiPartUpload(fileAccess: FileAccess, totalChunks: Int, chunkNumber: Int): UploadResult {
var uploadResult = UploadResult()
var uploadFile = File(transferItemModel.path)
val token = HttpRequestHelper.getToken()
var offset = chunkNumber.toLong() -1
BNLog.d(TAG, "offset:" + offset)
var fileByte = fileAccess.getBlock(offset * FileAccess.CHUNK_SIZE, uploadFile)
var mutipartBody = MultipartBody.Builder()
mutipartBody.setType(MultipartBody.FORM)
mutipartBody.addFormDataPart("fileFolderId", transferItemModel.fileFolderId)
.addFormDataPart("uploadId", uploadId)
.addFormDataPart("totalChunks", totalChunks.toString())
.addFormDataPart("chunkNumber", chunkNumber.toString())
.addFormDataPart("isAutoComplete", 1.toString())
.addFormDataPart("identifier", MultiPartUtil.getBytesMd5(fileByte))
sumByte = fileByte.size +sumByte
BNLog.d(TAG, "multiPartUpload 上传字节大小:" + fileByte.size +"当前已经上传的总大小:" +sumByte)
var body = RequestBody.create(MultipartBody.FORM, fileByte)
mutipartBody.addFormDataPart("file", uploadFile.name, body)
var request = Request.Builder().url(HttpPathEntity.HOST + HttpPathEntity.uploadPart.path)
.header("token", token)
.header("sign", HttpRequestHelper.DEFAULT_SIGN)
.header("deviceId", DeviceUtils.androidID)
.header("timestamp", System.currentTimeMillis().toString())
.post(mutipartBody.build())
.build()
var client = OkHttpClient()
call = client.newCall(request)
try {
var response =call.execute()
var responseBody = response.body()
if (response.isSuccessful) {
var content = response.body()?.string()
var result = JSONObject(content)
var resultCode = result.getInt("code")
var message = result.getString("message")
uploadResult.code = resultCode
uploadResult.msg = message
BNLog.d(TAG, " success responseBody:" + responseBody +"content:" + content)
}else {
BNLog.e(TAG, " fail responseBody:")
}
}catch (e: Exception) {
}
return uploadResult
}
suspend fun uploadFile() {
BNLog.d(TAG, "uploadFile filePath = " +transferItemModel.path +"threadName:" + Thread.currentThread().name)
var fileAccess = FileAccess(transferItemModel.path)
var totalChunks = fileAccess.chunkSize
var retryTimes =DEFAULT_RETRY_TIME
var isOk =false
var successCount =0
var totalFileByteSize = MultiPartUtil.getFileByteSize(transferItemModel.path)
BNLog.e(TAG, "文件的总长度:" +transferItemModel.fileSize +" 读取的总字节:" + totalFileByteSize)
while (retryTimes >0 && !isOk) {
for (iin 1 until (totalChunks +1)) {
BNLog.d(TAG, "uploadFile i=" + i)
if (i == totalChunks) {
BNLog.d(TAG, "上传最后一块:" + (totalChunks -1))
}else {
// continue
}
var uploadResult = multiPartUpload(fileAccess, totalChunks, i)
if (uploadResult.code == UploadResult.SUCCESS_CODE) {//上传成功
successCount++
var currentSize = successCount * FileAccess.CHUNK_SIZE//todo
var progress = (100 * currentSize / totalFileByteSize).toInt()
BNLog.e(TAG, "progress:" + progress)
if (progress >100) {
progress =100
}
this.ossListener?.onProgress(currentSize.toLong(), totalFileByteSize, progress)
}
// return//test
}
isOk = checkIsUploadSuccess(totalChunks, successCount)
isOk =true//test
retryTimes--
if (isOk) {
BNLog.d(TAG, "uploadFile 全部上传成功")
break
}
}
retryTimes =0
//上传完成,判断是否有失败的块
if (isOk) {
BNLog.e(TAG, "一个文件上传成功:" +transferItemModel.path)
this.ossListener?.onSuccess()
this.ossListener?.onReport(true)
var baonengId = UserCenterManage.getInstance(CloudServiceApp.getInstance()).getRefreshToken()
var uuid =transferItemModel.uuid
var uuidserver = baonengId +"/" + uuid
syncFileMeta(transferItemModel.fileFolderId, transferItemModel.name, uuidserver, transferItemModel.fileSize.toString(), transferItemModel.md5, transferItemModel.mineType)
}else {
this.ossListener?.onFailure(999, "fail")
}
}
/***
* 是否所有的块都成功了
*/
fun checkIsUploadSuccess(totalChunks: Int, successCount: Int): Boolean {
if (successCount >= totalChunks) {
return true
}
return false
}
okhttp相关的:
问题:
1.okhttp取消请求有回调吗?
2.multipart/form-data是浏览器提交表单上传文件的一种方式。
.addFormDataPart("uploadfile", uploadfile, RequestBody.create(MediaType.parse("*/*"), file)) // 第一个参数传到服务器的字段名,第二个你自己的文件名,第三个MediaType.parse("*/*")和我们之前说的那个type其实是一样的
3、为什么response.body().string() 只能调用一次
我们可能习惯在获取到Response对象后,先response.body().string()打印一遍log,再进行数据解析,却发现第二次直接抛异常,其实直接跟源码进去看就发现,通过source拿到字节流以后,直接给closeQuietly悄悄关闭了,这样第二次再去通过source读取就直接流已关闭的异常了。
publicfinal Stringstring()throws IOException{BufferedSource source=source();try{Charset charset=Util.bomAwareCharset(source,charset());returnsource.readString(charset);}finally{//这里讲resource给悄悄close了Util.closeQuietly(source);}}
解决方案:1.内存缓存一份response.body().string();2.自定义拦截器处理log。